Method for decoding HDR content

Abstract

A method for distributing high dynamic range (HDR) content to a playback device for displaying images, wherein the HDR content is encoded into an HDR bitstream and the HDR bitstream is subsequently decoded by the playback device. The HDR bitstream contains auxiliary metadata packets based on the processing capabilities of the playback device.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 046,796, filed on July 1, 2020, and European Patent Application No. 20183398.5, filed on July 1, 2020, both of which are incorporated herein by reference in their entirety. background 1. Technical Field

[0004] The present invention relates to high dynamic range images. More particularly, embodiments of the present invention relate to transmitting high dynamic range image content to playback devices having different levels of processing capabilities. 2. Background technology

[0006] As used herein, the term 'dynamic range (DR)' may relate to the ability of the human visual system (HVS) to perceive a range of intensities (e.g., luminance, luma) in an image, e.g., from the darkest gray (black) to the brightest white (highlight). In this sense, DR is related to 'scene-referred' intensities. DR may also relate to the ability of a display device to adequately or approximately render a range of intensities of a particular breadth. In this sense, DR is related to 'display-referred' intensities. Unless a particular sense is expressly specified at any point in the description herein to have a particular meaning, it should be inferred that the terms may be used in either sense, e.g., interchangeably.

[0007] As used herein, the term "high dynamic range (HDR)" refers to a DR width that spans approximately 14 to 15 orders of magnitude across the human visual system (HVS). In practice, the DR width over a wide range of intensities that humans can perceive simultaneously may be slightly truncated relative to HDR. An HDR playback system is a system that facilitates the display of HDR images. In a video display system, a typical feature of an HDR video playback system is that the color range and contrast of the video image display are improved compared to a standard video playback system.

[0008] Most consumer desktop monitors currently support 200 to 300 cd / m 2 Most consumer HDTVs range from 300 to 500 nits, with newer models reaching 1,000 nits (cd / m 2). Thus, such conventional displays represent a lower dynamic range (LDR) associated with HDR, also referred to as standard dynamic range (SDR). As the availability of HDR content increases due to the development of both capture devices (e.g., cameras) and HDR displays (e.g., Dolby Laboratories' PRM-4200 Professional Reference Monitor), HDR content can be color graded and displayed on HDR displays that support a higher dynamic range (e.g., from 1,000 nits to 5,000 nits or more). In general, the methods of the present disclosure relate to any dynamic range above SDR.

[0009] WO 2006 / 050305 A1 discloses a method and system for hosting and distributing enhanced color space content for different display devices (target color spaces) with display capabilities exceeding CRT color spaces. (Multiple) content creators establish basic or reference color space data and enhanced color space data for each target color space. The enhanced color space data is stored as metadata and transmitted separately from the basic / reference color space through an enhanced color channel. Both the basic / reference data and metadata are encoded before transmission and decoded on the consumer side by a separate decoder or a display device with an integrated decoder. Auxiliary data related to the target color space (such as brightness, brightness, contrast and other display settings) can be transmitted through the enhanced color channel and decoded to control the target color space settings including display settings. Color adjustment metadata can be included in the auxiliary data or maintained separately from the auxiliary data.

[0010] US2017 / 064334 A1 discloses a method for encoding a digital video, the method comprising: receiving a high dynamic range (HDR) master of the video, a reference standard dynamic range (SDR) master of the video, and target SDR display properties at an encoder; finding a color volume transform that converts HDR values ​​from the HDR master to SDR values, which are substantially similar to the SDR values ​​in the reference SDR master when converted for display on a target SDR display; converting the HDR values ​​from the HDR master to SDR values ​​using the color volume transform; generating a metadata item identifying the color volume transform to a decoder; and encoding the SDR values ​​into a bitstream.

[0011] US2018 / 077453 A1 discloses a device and method for transmitting / receiving a broadcast signal including video data and high dynamic range information. The broadcast signal transmission method may include the following steps: generating video data, generating a broadcast signal including the generated video data and image quality improvement metadata, and transmitting the generated broadcast signal.

[0012] EP 3 367 684 A1 discloses a method and apparatus for reconstructing an HDR image by applying a reconstruction process to an SDR image, the content of the SDR image being similar to the content of the HDR image, but the dynamic range of the luminance values ​​of the SDR image being lower than the dynamic range of the luminance values ​​of the HDR image, the reconstruction process requiring parameters obtained from a bitstream. The method is characterized in that the method further comprises determining whether all required parameters are available from the bitstream and recovering lost or damaged parameters from additional data, the reconstruction process further taking into account the recovered parameters. Summary of the invention

[0013] An embodiment of the present invention is based on a method for distributing high dynamic range (HDR) content to a playback device for displaying an image based on the HDR content, the method comprising: encoding the HDR content into an HDR bitstream, wherein encoding the HDR content comprises: mapping an HDR image in the HDR bitstream to one or more specified color volume base layers; generating one or more main metadata packets based on the HDR content; generating one or more auxiliary metadata packets, wherein the information in the auxiliary metadata packets is based on the capability of the playback device; encoding the one or more specified color volume base layers into one or more specified coded base layers; and combining the one or more specified coded base layers, the one or more main metadata packets and the one or more auxiliary metadata packets to generate an HDR bitstream; and decoding the HDR bitstream, wherein decoding the HDR bitstream comprises: selecting information from the one or more main metadata packets and the one or more auxiliary metadata packets based on the capability of the target playback device; processing the one or more coded base layers in the HDR bitstream based on the selected information to generate an image for display on the target playback device.

[0014] An embodiment of the present invention is based on a method for encoding HDR content into an HDR bitstream, wherein the method includes: mapping an HDR image in the HDR content to one or more specified color volume base layers, electro-optical transfer function (EOTF) base layers, and peak brightness base layers; generating one or more main metadata packets based on the HDR content; generating one or more auxiliary metadata packets, wherein the information in the auxiliary metadata packets is based on the capabilities of one or more playback devices; encoding the one or more specified color volume base layers, EOTF base layers, and peak brightness base layers into one or more specified coded base layers; and combining the one or more specified coded base layers, the one or more main metadata packets, and the one or more auxiliary metadata packets to produce an HDR bitstream.

[0015] An embodiment of the present invention is based on a method for decoding an HDR bitstream, wherein the HDR bitstream includes one or more coded base layers, one or more primary metadata packets and one or more auxiliary metadata packets, and the method includes: selecting information from the one or more primary metadata packets and the one or more auxiliary metadata packets based on the capabilities of a target playback device; processing the one or more coded base layers in the HDR bitstream based on the selected information to generate an image for display on the target playback device. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Embodiments of the invention are illustrated by way of example and not limitation in the accompanying drawings and in which like reference numerals refer to similar elements and in which:

[0017] Figure 1 A block diagram illustrating an exemplary implementation of an HDR bitstream encoding process.

[0018] Figure 2 A block diagram of an exemplary process for generating an image from an HDR bitstream using a playback process is shown.

[0019] Figure 3 A block diagram of an encoder employing both adding auxiliary metadata packages and optimizing the base layer is shown.

[0020] Figure 4 A block diagram of a low complexity playback process for generating an image from an HDR bitstream containing ancillary metadata packets is shown.

[0021] Figure 5A and Figure 5B An example decision tree regarding application of playback processing in a playback device is illustrated. DETAILED DESCRIPTION

[0022] Current HDR systems are mostly targeted at high-end playback devices. However, lower-end playback devices, such as those seen in mobile systems, will benefit from HDR content. Lower-end playback systems are often characterized by being able to perform only minimal or possibly no processing due to limitations of the processors in the system. Embodiments of the present invention provide for the delivery of HDR content to a mixed range of playback devices ranging from very low to very high performance.

[0023] As used herein, "metadata" refers to data that characterizes or relates to a bitstream, the images produced, and / or the equipment used in converting the bitstream to the produced images. As used herein, "primary metadata" refers to data that describes the characteristics of an HDR image, as well as synthesizer data. As used herein, "auxiliary metadata" refers to data that describes the capabilities of a playback device, in addition to display mapping data for an image.

[0024] Figure 1 A block diagram of an exemplary implementation of an encoding process is shown. Figure 1 In the embodiment, an HDR image 121 and any "HDR image metadata" 123 are input to an encoder 130. The encoder 130 includes a first pre-processing step 131 and a second encoding step 135. In an embodiment, the encoding step 135 may include a video compression step, such as is known in the art (e.g., AVC, HEVC, AV1, etc.). The pre-processing step 131 performs a mapping 133 to a selected color volume to generate a base layer image 136, and simultaneously generates 132 compositor metadata 139 that allows a playback device to restore the base layer mapping to reconstruct the original HDR image. Typically, the base layer mapping is selected to enable compatibility with certain playback devices, such as "SDR", "HDR10" or "HLG" devices. In addition to any HDR image metadata 123, the compositor metadata 139 is combined to create a master metadata packet 138. The encoding step generates an encoded base layer image 137 from the base layer image 136. The combination of the encoded base layer image 137 and the master metadata packet 138 comprises a bitstream 140 (e.g., Dolby Vision). TM bitstream).

[0025] Figure 2 A block diagram of an exemplary process for generating an image from a bitstream 240 using a playback process 260 is shown. Figure 2 As shown, in the playback process 260, the decoder and compositor step 242 first decodes the coded base layer image 237 and then applies the compositor metadata from the main metadata package 238 to reconstruct the original HDR image 221. Then, the display mapping step 245 maps the HDR image 221 to the target display 250 under the guidance of the HDR image metadata 223. The decoder and compositor process 242 corresponds to Figure 1 The encoding process 135 shown is compatible with the decoding process.

[0026] Playback device operations (such as Figure 2 The operations shown) may require significant processing power, first when applying the decoder and compositor process 242, and second when applying the display mapping process 245; the processing power required for both may vary from frame to frame.

[0027] The discussion of Dolby Vision and Dolby Vision bitstreams presented above provides an overview of the encoding and decoding process of HDR images. It should be understood by those skilled in the art that other processes may be used to encode and decode HDR images for display on a target display.

[0028] According to an embodiment of the present invention, playback complexity can be simplified by 1) adding auxiliary metadata packages and 2) optimizing the base layer image for mapping to a target display of a level. The addition of auxiliary metadata packages can be selected based on the processing capabilities of various playback devices. At least one of these auxiliary metadata packages contains only static compositors (enabling the playback device to bypass the compositor) and display mapping (DM) metadata describing the color volume and signal encoding of the base layer. Additional auxiliary metadata packages can be selected based on the playback device capabilities to add increased levels of functionality to the full metadata package. The key to optimizing the base layer is to perform a large amount of dynamic high-quality mapping during encoding (132) so that the playback device requires fewer secondary mappings (245). Therefore, the complexity of this secondary mapping can be reduced, making it possible to run on more low-capability playback devices. In one embodiment, the secondary mapping can be static.

[0029] Figure 3 A block diagram of an exemplary encoder 300 employing both adding auxiliary metadata packets and optimizing the base layer is shown. Figure 3 3, an HDR image 321 and HDR image metadata 323 are input to an encoder 300. The encoder 300 includes a first pre-processing step 331 and a second encoding step 335, the second encoding step including compression or other processing to generate an encoded optimized base layer image 337 for a bitstream 340. The pre-processing step 331 may perform an optimization mapping 333 on the optimal color rendering volume to generate an optimized base layer image 336. The pre-processing step 332 may be performed in a manner similar to that described above with respect to Figure 1 The preprocessing step 332 also generates one or more auxiliary metadata packets 339. The encoding step 335 generates an encoded optimized base layer image 337 from the optimized base layer image 336. The encoding step 335 can generate one or more content versions based on the same original HDR image in a manner described in additional detail below. The combination of the encoded optimized base layer image 337, the main metadata packet 338 and the auxiliary metadata packets 339 constitute a bitstream 340 configured for playback on a mixed range playback device.

[0030] A playback device receiving a bitstream with ancillary data packets can be configured to select between primary or ancillary metadata packets based on the processing capabilities of the device. These capabilities may be due to hardware limitations of the device, or they may be selected dynamically based on the availability of processing power or battery power. Very capable playback devices can follow the Figure 1 The operation shown in the figure, while low-complexity playback devices can use auxiliary metadata packages to simplify the playback process (see, for example Figure 4 ).

[0031] Figure 44 shows a block diagram of an exemplary low complexity playback process for generating an image from a bitstream 440 including a primary metadata packet and an auxiliary metadata packet 431. Figure 4 As shown, the playback process has a decoder step 442 that decodes the encoded optimized base layer image 437 into the optimized base layer image 436. Then, a display mapping step 445 maps the optimized base layer image 436 to the target display 480 under the guidance of the auxiliary metadata 439. Figure 4 In the low complexity embodiment shown, no synthesizer (such as Figure 2 Instead, the optimized base layer image 436 is mapped to the output 480 based solely on the auxiliary metadata package 439. In one embodiment, the auxiliary metadata package remains static for the duration of the content, thus requiring minimal dynamic processing.

[0032] In some embodiments (e.g., Dolby Vision), there may be multiple versions of the same content encoded with different encoding settings. For example, the encoder can be configured to generate two or more bitstreams intended for different playback devices. The difference between these versions is the compatibility of the base layer.

[0033] For example, there may be a "most compatible" base layer whose color volume corresponds to SDR (standard dynamic range). This base layer can be viewed on any legacy SDR playback device that does not have HDR capabilities. In contrast, HDR playback devices may apply both "compositors" and "mapping" to optimally display the image, such as Figure 2 According to the present invention, the HDR playback device may alternatively use an auxiliary metadata package to map the SDR signal directly to the target device capabilities (e.g. Figure 4 ).

[0034] There may also be a "least compatible" base layer of the original HDR image whose color volume corresponds to the original HDR image. This base layer is not directly viewable on SDR playback devices. HDR playback devices can use the Figure 2 Configured to process the base layer with the compositor and then display mapping.

[0035] There may also be a "medium compatible" base layer between the extremes of the "most compatible" base layer and the "least compatible" base layer. The color volume of this medium base layer can be configurable. In a preferred embodiment, the color volume is configured to be an average of the color volumes of the target playback devices. In one embodiment, the base layer color volume corresponds to P3 primaries and 200-400 nits peak luminance. If the characteristics of the playback device match the characteristics of the base layer, then legacy (non-HDR) playback devices may be able to correctly and optimally display the decoded image. Fully capable playback devices can use the master metadata package to apply the compositor and mapping process to optimally render the device (e.g. Figure 2 ), while limited-capability playback devices can use auxiliary metadata package decoding and apply limited display mapping processes to achieve image quality between the two (e.g. Figure 4 ).

[0036] In embodiments according to the invention, the bitstream may contain both primary and secondary metadata. The choice of compatibility is up to the user. Typically, "most compatible" reaches the largest number of devices, while "least compatible" has the best image quality only for compatible devices (or, the original HDR image that deviates the most from the SDR image). Embodiments of the invention with configurable base layers and secondary metadata provide image quality that can be between the most compatible and least compatible configurations. Note that depending on the color volume and peak brightness values ​​of the playback device, such as Figure 2 The playback system 260 shown may have slight quality differences in reconstructing HDR images when using the "most compatible", "medium compatible" or "least compatible" content types.

[0037] Embodiments of encoders according to the present invention may be configured to generate one or more content versions based on the same original HDR image, including a "most compatible" content version, a "medium compatible" content version, and a "least compatible" content version. The "most compatible" content is obtained by automatically mapping from HDR to a compatible base layer SDR. The most compatible content also includes metadata used by the playback device to reconstruct the original HDR and perform mapping from HDR to the target display. The color volume of the base layer can be 100 nits, R709, to maximize compatibility with playback devices that expect SDR content. The "medium compatible" content is obtained by automatically mapping from HDR to an incompatible base layer (referred to herein as a "middle dynamic range" base layer or MDR). The color volume of the base layer may be configurable. In one embodiment, the color volume is configured to be the average value of the color volume of the target playback device. In another embodiment, the base layer color volume corresponds to P3 primary colors and 200 nits peak brightness. The "least compatible" version is essentially an HDR image as described above.

[0038] It should also be noted that the encoder can also be configured to create a completely non-HDR version of the content, without any metadata. This latter version is usually obtained using a different processing flow independent of the HDR image.

[0039] According to an embodiment of the present invention, a bitstream contains both a main metadata packet and one or more auxiliary metadata packets. The main metadata packet may contain: compositor metadata, which, when applied by the compositor to the base layer, enables the compositor to reconstruct the original HDR version as closely as possible; and display mapping metadata corresponding to the reconstructed HDR, which, when applied to the reconstructed HDR image by the display mapping process, enables the display mapping process to map the reconstructed HDR to the capabilities of the target device. The auxiliary metadata packet may include: an identifier describing the processing capabilities of the playback device; a signal to bypass the compositor; and display mapping metadata corresponding to the mapped base layer, which, when applied to the base layer by the display mapping process, enables the display mapping process to map the base layer to the capabilities of the target device.

[0040] Different content versions including their associated metadata can be encoded using standard codecs, which may include adaptive bitrate ladders, and made available for distribution.

[0041] As noted above, various playback devices may be equipped with the same bitstream. The final image quality depends on the capabilities of the playback device and the availability of different content versions. Embodiments of the present invention provide consistency in playback image quality between the various playback devices.

[0042] According to an embodiment of the present invention, a method for distributing HDR content to a playback device to display an image based on the HDR content is provided. Examples of distributed content include streaming media, video on demand, broadcasting, digital file transfer, etc. Examples of playback devices include computers, televisions, tablet computers, smart phones, projectors, etc. The method may include encoding HDR content into an HDR bitstream. Encoding may include mapping an HDR image in an HDR bitstream to a specified color volume base layer, and then the specified color volume base layer may be encoded as a coded base layer. For example, the HDR image may be mapped to and encoded as an SDR base layer and / or an MDR base layer. Encoding may also include generating primary metadata and / or auxiliary metadata as corresponding metadata packets. The primary metadata may be based on the HDR content, and the auxiliary metadata may be based on the base layer. The coded base layer and the primary metadata and the auxiliary metadata are combined into a bitstream. The method may also include decoding the HDR bitstream, which may include selecting information from a primary metadata packet or an auxiliary metadata packet or a combination of the two.

[0043] Figure 5A and Figure 5B is an example of a decision tree regarding application of playback processing according to an embodiment of the present invention. Figure 5A and Figure 5B It shows how to improve the image quality according to the playback capability and content type. Note, but not limited to, Figure 5AThe decision tree shown in the figure only addresses the case where there is a single asset distribution, while Figure 5B Only the distribution of two assets, the most compatible bitstream and the medium compatible bitstream, is addressed.

[0044] As used herein, "basic SDR" refers to an SDR base layer that has no metadata associated with it, or has a minimal set of metadata (e.g., a device only supports 3rd party metadata for basic functionality).

[0045] As used herein, "primary metadata" (or PRIME) refers to metadata that utilizes dynamic processing (eg, dynamic multivariate multiple regression and dynamic display management) on a display device.

[0046] As used herein, "auxiliary metadata" (or AUX) refers to metadata that is selected when static processing is supported at the decoder and dynamic processing is not used on the display device.

[0047] As used herein, an "asset" refers to a base layer that is provided with PRIME and AUX metadata. An "SDR asset" refers to a base layer most compatible asset. An "MDR asset" refers to a base layer moderately compatible asset. A bitstream may include multiple assets that share the same metadata at the base layer. An example of an SDR asset is DOLBY TM Profile P9.2. An example of an MDR asset is DOLBY TM Profile P32.

[0048] In some embodiments, the base layer of the asset is derived from an HDR workflow. In some embodiments, the base layer of the asset is derived solely from an HDR workflow.

[0049] An example of asset usage could be a user generated content (UGC) provider that only provides SDR assets in the bitstream, thereby providing the greatest reach (most compatible with devices) at the cost of low bitrate. Another example of asset usage could be a premium content provider that provides both SDR and MDR assets in the bitstream, thereby providing the greatest reach (by including the most compatible assets) while providing higher quality to high-end devices (by including moderately compatible assets).

[0050] Figure 5A An example flow diagram of a system according to one embodiment is shown. Whether the playback device supports a compatible application for bitstream processing (e.g., the device can play back Dolby Vision TMA determination is made 511 on the playback device (bitstream) for the MDR asset. A "compatible application" is an application on the playback device specifically designed to play back the encoded bitstream described herein. If not supported, the basic SDR base layer (547) is decoded and rendered 555 to generate an image as an SDR image. Any available basic metadata information is used for processing. If the application is supported 511, a second determination is made 521 as to whether an MDR asset is present in the bitstream.

[0051] If an MDR asset is present 521, a determination is made 531 whether the device supports dynamic processing. Then, if dynamic processing is supported, high power decoding and rendering operations 551 are performed on the MDR asset and PRIME metadata 542 to generate an image using dynamic display mapping (e.g., dynamic display management - dynamic DM) and dynamic compositor prediction (e.g., dynamic multivariate multiple regression - dynamic MMR) 551 (see, e.g., Figure 2 high power consumption processing).

[0052] If an MDR asset is present 521 and the device does not support 531 dynamic processing, a static decode and render operation 553 is performed on the MDR asset with 541 the AUX metadata to generate an image (see e.g. Figure 4 low-power processing).

[0053] If an MDR asset is not present 521 (or is not supported by the device), and the device supports 533 dynamic processing, then an SDR asset 552 with PRIME metadata may be used to produce a dynamically processed SDR image.

[0054] If an MDR asset is not present 521 and the device does not support 533 dynamic processing, then an SDR image may be produced with static processing 554 using 544 an SDR asset with AUX metadata.

[0055] In this way, a wide range of devices can be used while still providing ever-increasing image quality for devices that support dynamic processing.

[0056] Figure 5B An example flow chart of a system according to another embodiment is shown. The flow chart is similar to Figure 5A , adds a decision to check whether an SDR asset exists 513, 323. If the device does not support 511 compatible applications, but the SDR asset exists 513, the SDR asset can be rendered 555 as SDR.

[0057] If the device does support 511 the application but the MDR asset does not exist 521 , the SDR asset does exist 523 , and the device supports 533 dynamic processing, then the SDR BL and PRIME metadata can be used 543 to produce an SDR image rendered 552 with dynamic MMR and dynamic DM.

[0058] If the device does support the application 511 but the MDR asset does not exist 521, the SDR asset does exist 523, and the device does not support dynamic processing 533, then the SDR BL and AUX metadata can be used 544 to produce 554 the SDR image with a static DM.

[0059] If the device does support the application 511 , but the MDR asset does not exist 521 and the SDR asset does not exist 523 , then the base SDR may be used 547 to decode and render 555 to SDR with any available base metadata.

[0060] The decision branch for the least compatible bitstream (e.g., HDR asset) is Figure 5A and Figure 5B The decision branch for the least compatible bitstream will branch further to the left (ie, should generally have higher quality) and follow similar logic, except that the least compatible bitstream also needs to support dynamic playback capabilities.

[0061] exist Figure 5A and Figure 5B , the decision to use only the primary metadata package (PRIME) or the auxiliary (AUX) metadata package is binary. However, the playback device can also be configured to select additional auxiliary metadata packages that the playback device can support in combination with the primary metadata. For example, a playback device may have only static synthesizer and static display mapping capabilities, but the device may also have dynamic resizing, frame rate letterbox filling / cropping capabilities, or other functions. In addition, the decision for processing can be based on values ​​in the configuration file, which can be specified by the processing capabilities in the playback device. The decision for processing can also be based on other factors, such as the power level of the playback device, other requirements on the processing capabilities of the playback device, user preferences, and additional content metadata.

[0062] In practice, playback devices may not use all of the assets provided, but there is still value in allowing MDR and SDR assets to coexist. First, MDR assets may not be sufficient for full compatibility with target devices. However, users with high-end and mid-range devices will expect better quality than what is provided by SDR assets alone. Since the compositor has less mapping between the MDR BL and the HDR image, including MDR assets can provide better quality even for mid-range devices that only support static mapping. Note that MDR assets have fewer constraints since the base layer is not expected to match the SDR deliverable exactly.

[0063] For the example of premium content with both SDR assets and MDR assets, there are at least three possibilities:

[0064] 1. The device has a compatible application and supports HDR and dynamic processing -> MDR assets are used with dynamic processing (using PRIME),

[0065] 2. The device has a compatible app and supports HDR, but does not support dynamic processing -> use MDR assets, no dynamic processing (using AUX), or

[0066] 3. The device does not have a compatible application (e.g., competitor's device) -> SDR assets are used with generic (basic) metadata.

[0067] On the device side, for the example of a user having an HDR-compatible device with a compatible app and supporting dynamic processing, there are at least six possible outcomes:

[0068] 1. Basic SDR stream available (e.g. user generated content without PRIME / AUX compatible encoding) -> Use SDR + basic metadata,

[0069] 2. Old technology SDR assets available (PRIME only) -> Use SDR+PRIME,

[0070] 3. New technology SDR assets available (PRIME and AUX) -> Use SDR+PRIME,

[0071] 4. New technology SDR assets are available (PRIME and AUX), but the device is in low power mode -> use SDR+AUX, or

[0072] 5. New technical MDR assets are available (e.g. premium content) -> use MDR+PRIME, or

[0073] 6. New technology MDR assets are available (e.g. premium content), but device is in low power mode -> use MDR+AUX.

[0074] Example embodiments related to high dynamic range image distribution have been described herein with reference to Dolby Vision or Dolby Vision bitstreams as examples of HDR bitstreams. It will be appreciated by those skilled in the art that HDR image data may be included in an HDR bitstream other than a Dolby Vision bitstream. Embodiments of the present invention may be configured to operate with and support other HDR bitstreams.

[0075] Embodiments of the present invention may be implemented using a computer system, a system configured with electronic circuits and components, an integrated circuit (IC) device such as a microcontroller, a field programmable gate array (FPGA) or another configurable or programmable logic device (PLD), a discrete time or digital signal processor (DSP), an application specific IC (ASIC), and / or an apparatus including one or more of such systems, devices, or components. The computer and / or IC may execute, control, or implement instructions related to block-based reshaping of images with enhanced dynamic range, such as those described herein. The computer and / or IC may calculate any of the various parameters or values ​​related to the distribution of high dynamic range images described herein. The image and video embodiments may be implemented in hardware, software, firmware, and various combinations thereof.

[0076] Certain embodiments of the present invention include a computer processor that executes software instructions that cause the processor to perform the method of the present invention. For example, one or more processors in a display, an encoder, a set-top box, a transcoder, etc., can implement the method related to the distribution of high dynamic range images as described above by executing software instructions in a program memory accessible to the processor. The present invention can also be provided in the form of a program product. The program product can include any non-transient medium carrying a set of computer-readable signals, and the set of computer-readable signals includes instructions that cause the data processor to perform the method of the present invention when executed by a data processor. According to the program product of the present invention, any of various forms can be used. The program product can include, for example, a physical medium, such as a magnetic data storage medium including a floppy disk, a hard disk drive, an optical data storage medium including a CD ROM, a DVD, an electronic data storage medium including a ROM, a flash RAM, etc. The computer-readable signal on the program product can be optionally compressed or encrypted.

[0077] Where components (e.g., software modules, processors, components, devices, circuits, etc.) are mentioned above, unless otherwise indicated, references to the components (including references to "means") should be interpreted to include any components that perform the functions of the described components as equivalents (e.g., functionally equivalent) to the components, including components that are not structurally equivalent to the disclosed structures that perform the functions in the illustrated example embodiments of the invention.

[0078] As described herein, embodiments of the present invention may therefore be directed to one or more of the following enumerated example embodiments. Accordingly, the present invention may be implemented in any form described herein, including but not limited to the following enumerated example embodiments (EEE) that describe the structure, features, and functions of some parts of the present invention:

[0079] EEE1. A method for distributing high dynamic range (HDR) content to a playback device for displaying an image based on the HDR content, the method comprising: encoding the HDR content into an HDR bitstream, wherein encoding the HDR content comprises: mapping an HDR image in the HDR bitstream to a specified color volume base layer; generating a main metadata packet based on the HDR content; generating an auxiliary metadata packet, wherein information in the auxiliary metadata packet is based on a capability of the playback device; encoding the specified color volume base layer into a coded base layer; and combining the coded base layer, the main metadata packet and the auxiliary metadata packet to generate an HDR bitstream; and decoding the HDR bitstream, wherein decoding the HDR bitstream comprises: selecting information from the main metadata packet and the auxiliary metadata packet based on the capability of the playback device; and processing the coded base layer in the HDR bitstream based on the selected information to generate an image for display on the playback device.

[0080] EEE2. A method as described in enumerated example embodiment 1, wherein the main metadata package includes: synthesizer metadata and display mapping metadata.

[0081] EEE3. A method as described in enumerated example embodiments 1 or 2, wherein the auxiliary metadata package includes: an identifier describing the processing capabilities of the playback device; display mapping data; and a signal for bypassing synthesis operations when decoding an HDR bitstream.

[0082] EEE4. A method as described in any of the enumerated example embodiments 1-3, wherein the HDR bitstream includes a specified coded base layer, wherein the selectable specified coded base layer is one of: a most compatible base layer, the most compatible base layer comprising a standard dynamic range (SDR) color volume; a least compatible base layer, the least compatible base layer comprising a color volume corresponding to the color volume of the HDR image; and a medium compatible base layer, the medium compatible base layer comprising a color volume corresponding to a playback device.

[0083] EEE5. A method as described in enumerated example embodiment 4, wherein processing the coded base layer includes: determining a processing capability of a playback device, wherein the processing capability of the playback device is determined to be a processing capability that supports a processing capability selected from the following: SDR processing; a most compatible base layer processing or a medium compatible base layer processing; wherein, if the playback device only supports SDR processing, rendering the HDR bitstream as an image with an SDR color volume; wherein, if the playback device supports the most compatible base layer processing, determining whether the target playback device supports dynamic processing, wherein if the target playback device does not support dynamic processing, rendering the HDR bitstream with a static display mapping, and wherein, if the target playback device supports dynamic processing, decoding and rendering the HDR bitstream with a dynamic display mapping; and wherein, if the playback device supports medium compatible base layer processing, determining whether the target playback device supports dynamic processing, wherein if the target playback device does not support dynamic processing, rendering the HDR bitstream with a static display mapping, and wherein, if the target playback device supports dynamic processing, decoding and rendering the HDR bitstream with a dynamic display mapping.

[0084] EEE6. An apparatus comprising a processor and configured to perform a method as described in any one of the enumerated example embodiments 1 to 5.

[0085] EEE7. A non-transitory computer-readable storage medium having stored thereon computer-executable instructions for executing a method using one or more processors as described in any one of the enumerated example embodiments 1 to 5.

[0086] EEE8. A method for encoding high dynamic range (HDR) content into an HDR bitstream, wherein the method comprises: mapping an HDR image in the HDR content to a specified color volume base layer; generating a main metadata packet based on the HDR content; generating an auxiliary metadata packet, wherein the information in the auxiliary metadata packet is based on the capabilities of a playback device; encoding the specified color volume base layer into a specified coded base layer; and combining the specified coded base layer, the main metadata packet, and the auxiliary metadata packet to produce an HDR bitstream.

[0087] EEE9. A method as described in enumerated example embodiment 8, wherein the main metadata package includes: synthesizer metadata; and display mapping metadata; and wherein the auxiliary metadata package includes: an identifier describing the processing capabilities of the playback device; display mapping data; and a signal for bypassing synthesis operations when decoding an HDR bitstream.

[0088] EEE10. A method as described in enumerated example embodiments 8 or 9, wherein the HDR bitstream includes a specified coded base layer, wherein the specified coded base layer is one of: a most compatible base layer, the most compatible base layer including a standard dynamic range (SDR) color volume; a least compatible base layer, the least compatible base layer including a color volume corresponding to a color volume of an HDR image; and a medium compatible base layer, the medium compatible base layer including a color volume corresponding to a playback device.

[0089] EEE11. An apparatus comprising a processor and configured to perform a method as described in any one of enumerated example embodiments 8 to 10.

[0090] EEE12. A non-transitory computer-readable storage medium having stored thereon computer-executable instructions for performing a method using one or more processors as described in any one of enumerated example embodiments 8 to 10.

[0091] EEE13. A method for decoding an HDR bitstream, wherein the HDR bitstream includes a coded base layer, a primary metadata packet, and an auxiliary metadata packet, and the method includes: selecting information from the primary metadata packet and the auxiliary metadata packet based on the capabilities of a target playback device; and processing the coded base layer in the HDR bitstream based on the selected information to produce an image for display on the playback device.

[0092] EEE14. A method as described in enumerated example embodiment 13, wherein processing the coded base layer includes: decoding the coded base layer into a decoded base layer image; and mapping the decoded base layer image to a playback display based on data in the auxiliary metadata packet.

[0093] EEE15. A method as described in enumerated example embodiments 13 or 14, wherein processing the coded base layer includes: determining a processing capability of a playback device, wherein the processing capability of the playback device is determined to be a processing capability that supports a processing capability selected from the following: SDR processing; a most compatible base layer processing or a medium compatible base layer processing; and wherein, if the playback device only supports SDR processing, rendering the HDR bitstream as an image with an SDR color volume; and wherein, if the playback device supports the most compatible base layer processing, determining whether the target playback device supports dynamic processing, wherein if the playback device does not support dynamic processing, rendering the HDR bitstream with a static display mapping, and wherein, if the playback device supports dynamic processing, decoding and rendering the HDR bitstream with a dynamic display mapping; and wherein, if the playback device supports medium compatible base layer processing, determining whether the playback device supports dynamic processing, wherein if the playback device does not support dynamic processing, rendering the HDR bitstream with a static display mapping, and wherein, if the playback device supports dynamic processing, decoding and rendering the HDR bitstream with a dynamic display mapping.

[0094] EEE16. An apparatus comprising a processor and configured to perform a method as described in any one of enumerated example embodiments 13 to 15.

[0095] EEE17. A non-transitory computer-readable storage medium having stored thereon computer-executable instructions for performing a method using one or more processors as described in any of the enumerated example embodiments 13 to 15.

[0096] Thus, example embodiments involving the distribution of high dynamic range images have been described. In the foregoing description, embodiments of the invention have been described with reference to numerous specific details that may vary depending on the implementation. Therefore, the sole and exclusive indicator of the invention and the applicant's inventive intent is the set of claims issued from this application in the specific form in which the claims are issued. Any definitions expressly set forth herein for terms contained in such claims should govern the meaning of such terms as used in the claims. Therefore, limitations, elements, characteristics, features, advantages, or attributes that are not expressly cited in a claim should not limit the scope of such claim in any way. Therefore, the specification and drawings should be viewed in an illustrative rather than a restrictive sense.

Claims

1. A method for decoding high dynamic range (HDR) content at a playback device to display an image based on the HDR content, the method comprising: Receiving an HDR bitstream in which the HDR content is encoded, wherein the HDR bitstream comprises: a coded base layer, the coded base layer being coded from a specified color volume base layer, the HDR image in the HDR bitstream being mapped to the specified color volume base layer; A master metadata packet, wherein information in the master metadata packet is based on the HDR content, wherein the master metadata packet includes: compositor metadata which, when applied by a compositor to the specified color volume base layer, enables the compositor to reconstruct the original HDR content; and display mapping metadata corresponding to the reconstructed HDR content for dynamic display mapping, the display mapping metadata, when applied to the reconstructed HDR content by a dynamic display mapping process, enables the dynamic display mapping process to map the reconstructed HDR content to the capabilities of the playback device; Auxiliary metadata package, wherein the auxiliary metadata package includes: a static compositor and display mapping metadata corresponding to the specified color volume base layer for static display mapping, the static compositor and display mapping metadata, when applied to the specified color volume base layer by a static display mapping process, enabling the static display mapping process to map the specified color volume base layer to the capabilities of the playback device; an identifier describing the processing capabilities of the playback device required to apply the static compositor and display mapping metadata; and signal for bypassing a synthesis operation when decoding the HDR bitstream; and Additional auxiliary metadata packages based on playback device capabilities to add increased levels of functionality; as well as Decoding the HDR bitstream, wherein decoding the HDR bitstream comprises: upon determining that the playback device matches the required processing capabilities of the primary metadata package and the playback device supports one or more functions defined by the additional auxiliary metadata package, using the additional auxiliary metadata package in conjunction with the primary metadata package; and The coded base layer in the HDR bitstream is processed based on the selected metadata to produce an image for display on the playback device.

2. The method according to claim 1, wherein: The coding base layer is one of: A most compatible base layer, the most compatible base layer corresponding to a standard dynamic range (SDR) color volume; a least compatible base layer, the least compatible base layer corresponding to a color rendering volume corresponding to the color rendering volume of the HDR image; as well as A medium-compatible base layer corresponds to a color rendering volume corresponding to the playback device.

3. The method according to claim 1 or claim 2, wherein: Processing the coded base layer comprises: decoding the coded base layer into a decoded base layer picture; and The decoded base layer image is mapped to a playback display based on the data in the auxiliary metadata package.

4. The method according to claim 2, wherein: Processing the coded base layer comprises: Determining a processing capability of the playback device, wherein the processing capability of the playback device is determined to support a processing capability selected from the group consisting of: SDR processing; most compatible base layer processing or medium compatible base layer processing; Wherein, if the playback device only supports SDR processing, rendering the HDR bitstream into an image with an SDR color volume; wherein, if the playback device supports the most compatible base layer processing, determining whether the playback device supports dynamic processing, wherein if the playback device does not support dynamic processing, rendering the HDR bitstream with a static display mapping, and wherein, if the playback device supports dynamic processing, decoding and rendering the HDR bitstream with a dynamic display mapping; and wherein, if the playback device supports medium-compatible base layer processing, determining whether the playback device supports dynamic processing, wherein if the playback device does not support dynamic processing, rendering the HDR bitstream with a static display mapping, and wherein, if the playback device supports dynamic processing, decoding and rendering the HDR bitstream with a dynamic display mapping.

5. The method according to any one of claims 1 to 4, further comprising: Prior to receiving the HDR bitstream, encoding high dynamic range (HDR) content into an HDR bitstream, wherein the encoding comprises: Mapping the HDR image in the HDR content to a specified color rendering volume base layer; Generate a master metadata packet based on the HDR content, wherein the master metadata packet includes: compositor metadata which, when applied by a compositor to the specified color volume base layer, enables the compositor to reconstruct the original HDR content; and display mapping metadata corresponding to the reconstructed HDR content for dynamic display mapping, the display mapping metadata, when applied to the reconstructed HDR content by a dynamic display mapping process, enables the dynamic display mapping process to map the reconstructed HDR content to the capabilities of the playback device; Generate an auxiliary metadata package, wherein the auxiliary metadata package includes: a static compositor and display mapping metadata corresponding to the specified color volume base layer for static display mapping, the static compositor and display mapping metadata, when applied to the specified color volume base layer by a static display mapping process, enabling the static display mapping process to map the specified color volume base layer to the capabilities of the playback device; an identifier describing the processing capabilities of the playback device required to apply the static compositor and display mapping metadata; and signal for bypassing a synthesis operation when decoding the HDR bitstream; generating additional auxiliary metadata packages based on the playback device capabilities to add increased levels of functionality; encoding the specified color volume base layer as a coded base layer; and The coded base layer, the primary metadata package, the auxiliary metadata package and the additional auxiliary metadata package are combined to generate the HDR bitstream.

6. A decoding device comprising a processor and configured to execute the method according to any one of claims 1 to 4.

7. A system for encoding and decoding, comprising a decoding device according to claim 6 and an encoding device configured to perform the encoding according to claim 5.

8. A computer-readable storage medium having stored thereon computer-executable instructions for executing the method according to any one of claims 1 to 5 using one or more processors.

9. A computer program product, comprising computer executable instructions for executing the method according to any one of claims 1 to 5 using one or more processors.

Images