Techniques for pre-processing images to improve gain map compression results

Abstract

Techniques are provided for pre-processing images to improve gain map compression results. This application describes techniques for pre-processing images to improve gain map compression results. A gain map can be generated by comparing a first image to a second image and subsequently compressed to form a compressed gain map. The compressed gain map can be combined with a compressed version of the first image to form a compressed enhanced image. The compressed enhanced image can later be decompressed to generate an uncompressed version of the first image, and the gain map is applied to the uncompressed version of the first image to generate a version of the second image to provide playback on a target display. The compressed version of the first image and the compressed gain map can be generated separately by an image compression module and a gain map compression module, or together by a combined gain map generation and compression module.

Description

[Technical Field]

[0001]

[0003] Embodiments described herein describe techniques for pre-processing images to improve gain map compression results. A gain map may be generated by comparing a first image to a second image and subsequently compressed to form a compressed gain map. The compressed gain map may be combined with a compressed version of the first image to form a compressed enhanced image. The compressed enhanced image may later be decompressed, and the gain map may be applied to the first image to generate a version of the second image to provide playback on a target display. [Background technology]

[0002] The dynamic range of an image refers to the range of pixel values ​​between the brightest and darkest parts of the image (also called "luminance"). In particular, an image sensor captures a limited range of luminance in a single exposure of a scene, relative to human visual perception of the scene. Images with a limited range are referred to herein as standard dynamic range (SDR) images.

[0003] Despite the limitations of image sensors, improvements in computational photography allow a wider range of luminance values ​​to be captured by an image sensor using multiple images processed together to form an image with a wider range, referred to herein as a high dynamic range (HDR) image. An HDR image can be formed by (1) capturing multiple bracketed images, i.e., individual SDR images each captured using a different exposure value (also called "aperture"), and (2) merging the bracketed SDR images into a single HDR image that incorporates aspects from the different exposures. The single HDR image contains a wider dynamic range of luminance values ​​compared to the narrower range of luminance values ​​in each of the individual SDR images. HDR images can be considered superior to SDR images because more scene information is preserved by HDR images than by individual SDR images.

[0004] Display devices capable of displaying HDR images with a wider range of luminance values ​​are becoming more accessible due to advances in design and manufacturing technologies. However, the majority of display devices currently in use (and continue to be manufactured) are only capable of displaying SDR images with a more limited range of luminance values. As a result, HDR images must be converted (i.e., downgraded) to SDR image equivalents for display on devices with only SDR-capable displays. Conversely, devices with HDR-capable displays may attempt to convert (i.e., upgrade) SDR images to HDR images equivalent for display via an HDR-capable display.

[0005] Existing conversion techniques can produce inconsistent and / or undesirable results. In particular, downgrading an HDR image to an SDR image (which can be performed through a tone mapping operation) can introduce visual artifacts (e.g., banding) into the resulting SDR image, which are often uncorrectable with additional image processing. Conversely, upgrading an SDR image to an HDR image (which can be performed through an inverse tone mapping operation) involves applying various levels of inference, which can also introduce uncorrectable visual artifacts.

[0006] Furthermore, keeping all of the originally captured SDR images along with the HDR images may use up limited storage space and may require additional communication bandwidth to transfer all of the images between devices.

[0007] Therefore, what is needed are techniques that allow images to be efficiently and accurately converted between different states. For example, it would be desirable to allow an SDR image to be upgraded to an HDR counterpart (and vice versa) without relying on the aforementioned (and inadequate) conversion techniques. Summary of the Invention

[0008] Embodiments described herein describe techniques for pre-processing images to improve gain map compression results. A gain map can be generated by comparing a first image to a second image and subsequently compressed to form a compressed gain map. The compressed gain map can be combined with a compressed version of the first image to form a compressed enhanced image. The compressed enhanced image can later be decompressed, and the gain map applied to the first image to generate a version of the second image for playback on a target display. In some embodiments, the first image comprises a standard dynamic range (SDR) image and the second image comprises a high dynamic range (HDR) image. In some embodiments, the first image comprises an SDR image selected from a plurality of SDR images, and the second image comprises an HDR image derived from a combination of the plurality of SDR images. In some embodiments, the gain map is determined by comparing luminance values ​​of pixels in the HDR image to luminance values ​​of corresponding pixels in the SDR image. In some embodiments, the full-resolution version of the gain map includes a gain value for each pixel in the SDR image and the HDR image, and the reduced-resolution version of the gain map includes gain values ​​for groups of two or more pixels in the SDR image and the HDR image. In some embodiments, the gain map is determined at half or quarter resolution compared to the SDR image and the HDR image. In some embodiments, the compressed gain map is generated by processing the gain map (at full resolution or reduced resolution) using a first (gain map) compression scheme, and the compressed version of the first image is generated by processing the first image using a second (image) compression scheme. In some embodiments, the gain map is generated by comparing the luminance values ​​of the SDR image with the luminance values ​​of the HDR image. In some embodiments, the gain map is generated by comparing the luminance values ​​of a compressed version of the SDR image (or a compressed version of the HDR image) with the luminance values ​​of the (uncompressed) HDR image (or the uncompressed SDR image).In some embodiments, the compressed enhanced image is determined by determining a compressed version of an image derived from the SDR image and the HDR image together with a compressed version of the gain map (or determined together with an uncompressed version of the gain map that is then compressed), and the compressed version of the gain map is included with the compressed version of the image to form the compressed enhanced image. In some embodiments, the compressed enhanced image is stored on a non-volatile storage medium accessible locally by the computing device and / or accessible remotely by the computing device and possibly by other computing devices, for example, via a cloud network-based service. In some embodiments, a second computing device obtains the compressed enhanced image, extracts a compressed version of the image and a compressed version of the gain map from the compressed enhanced image, generates an uncompressed version of the image from the compressed version of the image, generates an uncompressed version of the gain map from the compressed version of the gain map, and applies the uncompressed version of the gain map to the uncompressed version of the image to generate a second image formatted for display by the second computing device. The second image has a dynamic range of luminance values ​​that differs from the dynamic range of luminance values ​​of the uncompressed version of the image. In some embodiments, the second image is an HDR image and the uncompressed version of the image is an SDR image. In some embodiments, the compressed version of the first image and the compressed version of the gain map are generated separately by an image compression module and a gain map compression module, or generated together by a combined gain map generation and compression module.In some embodiments, the computing device generates a gain map, compresses the gain map to form a compressed version of the gain map, decompresses the compressed version of the gain map to generate an uncompressed version of the gain map, compares the (original) gain map with the uncompressed version of the gain map to determine an error map, and stores the error map along with the compressed version of the gain map to be used in creating a second image formatted for display by a second computing device.

[0009] Other embodiments include a non-transitory computer-readable storage medium configured to store instructions that, when executed by a processor included in the computing device, cause the computing device to perform various steps of any of the aforementioned methods. Further embodiments include a computing device configured to perform various steps of any of the aforementioned methods.

[0010] Other aspects and advantages of the present invention will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, which illustrate, by way of example, the principles of the described embodiments.

[0011] The present disclosure will be readily understood by the following detailed description in conjunction with the accompanying drawings, in which like reference numerals indicate like structural elements, and in which: [Brief explanation of the drawings]

[0012] [Figure 1] 1 illustrates an overview of a computing device that can be configured to perform various techniques described herein, according to some embodiments.

[0013] [Figure 2A] 1 shows a sequence of conceptual diagrams of a technique for generating a gain map based on an SDR image and an HDR image, according to some embodiments. [Figure 2B]1 shows a sequence of conceptual diagrams of a technique for generating a gain map based on an SDR image and an HDR image, according to some embodiments. [Figure 2C] 1 shows a sequence of conceptual diagrams of a technique for generating a gain map based on an SDR image and an HDR image, according to some embodiments. [Figure 2D] 1 shows a sequence of conceptual diagrams of a technique for generating a gain map based on an SDR image and an HDR image, according to some embodiments. [Figure 2E] 1 shows a sequence of conceptual diagrams of a technique for generating a gain map based on an SDR image and an HDR image, according to some embodiments.

[0014] [Figure 3A] 1A-1C illustrate a sequence of conceptual diagrams for generating a compressed enhanced image based on a first image and a second image, according to some embodiments. [Figure 3B] 1A-1C illustrate a sequence of conceptual diagrams for generating a compressed enhanced image based on a first image and a second image, according to some embodiments. [Figure 3C] 1A-1C illustrate a sequence of conceptual diagrams for generating a compressed enhanced image based on a first image and a second image, according to some embodiments. [Figure 3D] 1A-1C illustrate a sequence of conceptual diagrams for generating a compressed enhanced image based on a first image and a second image, according to some embodiments. [Figure 3E] 1A-1C illustrate a sequence of conceptual diagrams for generating a compressed enhanced image based on a first image and a second image, according to some embodiments.

[0015] [Figure 4] 1 shows an example diagram of generating an HDR image for display by a computing device from a compressed enhanced image according to some embodiments.

[0016] [Figure 5A]1 illustrates a flowchart of an exemplary method for image management by a computing device, according to some embodiments. [Figure 5B] 1 illustrates a flowchart of an exemplary method for image management by a computing device, according to some embodiments.

[0017] [Figure 6] FIG. 1 shows a detailed diagram of a computing device that can be used to perform various techniques described herein, according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0018] Representative examples of applications of the methods and apparatus according to the present application are described in this section. These examples are provided solely to add context and to aid in understanding the described embodiments. Thus, it will be apparent to one skilled in the art that the described embodiments may be practiced without some or all of these specific details. In other instances, well-known process steps have not been described in detail to avoid unnecessarily obscuring the described embodiments. Other applications are possible, and therefore the following examples should not be construed as limiting.

[0019] In the following detailed description, reference is made to the accompanying drawings which form a part of the description, and in which is shown by way of illustration specific embodiments in accordance with the described embodiments. These embodiments are described in sufficient detail to enable one skilled in the art to practice the described embodiments, but it is to be understood that these examples are not limiting, and that other embodiments may be used, and changes may be made without departing from the spirit and scope of the described embodiments.

[0020] Representative embodiments described herein disclose techniques for generating a gain map based on acquired images. In particular, the gain map may be generated by comparing a first image with a second image. The first image and the gain map can then be used to efficiently reconstruct the second image by embedding the gain map into the first image. A more detailed description of these techniques is provided below in connection with FIGS. 1, 2A-2E, 3A-3E, 4, 5A, 5B, and 6.

[0021] FIG. 1 illustrates an overview 100 of a computing device 102 that can be configured to perform various techniques described herein. As shown in FIG. 1, the computing device 102 can include a processor 104, a volatile memory 106, and a non-volatile memory 124. A more detailed breakdown of exemplary hardware components that may be included in the computing device 102 is illustrated in FIG. 5; note that these components are omitted from the illustration of FIG. 1 solely for simplicity. For example, the computing device 102 can include additional non-volatile memory (e.g., a solid-state drive, a hard drive, etc.), other processors (e.g., a multi-core central processing unit (CPU)), a graphics processing unit (GPU), etc.). According to some embodiments, an operating system (OS) (not shown in FIG. 1) can be loaded into the volatile memory 106, and the OS can execute various applications that collectively enable the various techniques described herein to be implemented. For example, these applications can include an image analyzer 110 (and its internal components), a gain map generator 120 (and its internal components), one or more compressors (not shown in FIG. 1), etc.

[0022] As shown in FIG. 1 , the volatile memory 106 can be configured to receive a multi-channel image 108. The multi-channel image 108 can be provided, for example, by a digital imaging unit (not shown in FIG. 1 ) configured to capture and process digital images. According to some embodiments, the multi-channel image 108 can be comprised of a collection of pixels, with each pixel in the collection of pixels including a group of sub-pixels (e.g., red sub-pixels, green sub-pixels, blue sub-pixels, etc.). It should be noted that the term “sub-pixel” as used herein can be synonymous with the term “channel.” It should also be noted that the multi-channel image 108 can have different resolutions, layouts, bit depths, etc. without departing from the scope of this disclosure.

[0023] According to some embodiments, a given multi-channel image 108 may represent a standard dynamic range (SDR) image constituting a single exposure of a scene collected and processed by the digital imaging unit. A given multi-channel image 108 may also represent a high dynamic range (HDR) image constituting multiple exposures of a scene collected and processed by the digital imaging unit. To generate an HDR image, the digital imaging unit may capture the scene under different exposure brackets, for example, three exposure brackets often referred to as “EV0,” “EV−,” and “EV+.” Generally, an EVO image corresponds to a normal / ideal exposure of the scene (typically captured using the digital imaging unit's auto-exposure setting). An EV image corresponds to an underexposed image of the scene (e.g., four times darker than EV0), and an EV+ image corresponds to an overexposed image of the scene (e.g., four times brighter than EV0). The digital imaging unit may combine different exposures to generate a resultant image incorporating a wider range of brightness relative to an SDR image. Note that the multi-channel image 108 described herein is not limited to SDR / HDR images. Conversely, multi-channel image 108 may represent any form of digital image (eg, scanned image, computer-generated image, etc.) without departing from the scope of this disclosure.

[0024] 1 , the multi-channel image 108 may (optionally) be provided to an image analyzer 110. According to some embodiments, the image analyzer 110 may include various components configured to process / modify the multi-channel image 108 as needed. For example, the image analyzer 110 may include a tone mapping unit 112 (e.g., configured to perform global / local tone mapping operations, inverse tone mapping operations, etc.), a noise reduction unit 114 (e.g., configured to reduce global / local noise in the multi-channel image), a color correction unit 116 (e.g., configured to perform global / local color corrections on the multi-channel image), and a sharpening unit 118 (e.g., configured to perform global / local sharpening corrections on the multi-channel image). It should be noted that the image analyzer 110 is not limited to the aforementioned processing units, and the image analyzer 110 may incorporate any number of processing units configured to perform any processing / modifications on the multi-channel image 108 without departing from the scope of the present disclosure.

[0025] As shown in FIG. 1 , the multi-channel image 108 may be provided to the gain map generator 120 after being processed by the image analyzer 110. However, it should be noted that the multi-channel image 108 may bypass the image analyzer 110 and be provided to the gain map generator 120, if desired, without departing from the scope of this disclosure. It should also be noted that the multi-channel image 108 may bypass one or more of the processing units of the image analyzer 110 without departing from the scope of this disclosure. For example, two given multi-channel images may pass through the tone mapping unit 112 to receive local tone mapping modifications and then bypass the remaining processing units in the image analyzer 110. In this regard, the two multi-channel images that have undergone local tone mapping operations may be utilized to generate a gain map 123 that reflects the performed local tone mapping operations. In either case, as described in more detail herein, upon receiving the two multi-channel images 108, the gain map generator 120 may generate the gain map 123 based on the two multi-channel images 108. Gain map generator 120 can then store gain map 123 in one of the two multi-channel images 108 to generate enhanced multi-channel image 122. It is further noted that the gain map generation technique can be performed at any time relative to receipt of the multi-channel image on which the gain map is based. For example, gain map generator 120 can be configured to postpone generation of the gain map when the digital imaging unit is actively being used to ensure adequate processing resources are available so that no slowdown is imposed on the user. A more detailed breakdown of how gain map generator 120 can generate gain map 123 is provided below in connection with FIGS. 2A-2E, 3A-3G, and 4A-4E.

[0026] Additionally, although not shown in FIG. 1 , one or more compressors may be implemented on the computing device 102 to compress the enhanced multi-channel image 122. For example, the compressor may implement a Lempel-Ziv-Welch (LZW)-based compressor, other types of compressors, a combination of compressors, etc. Furthermore, the compressor may be implemented in any manner to establish the most efficient environment for compressing the enhanced multi-channel image 122. For example, multiple buffers may be instantiated (pixels may be pre-processed in parallel), each buffer may be coupled to a separate compressor, and the buffers may be compressed simultaneously in parallel. Furthermore, compressors of the same or different types may be coupled to each buffer based on the format of the enhanced multi-channel image 122.

[0027] 1 thus provides a high-level overview of different hardware / software architectures that may be implemented by computing device 102 to perform the various techniques described herein. A more detailed breakdown of these techniques is described below in conjunction with FIGS. 2A-2E.

[0028] 2A-2E show a sequence of conceptual diagrams of a technique for generating a gain map based on an SDR image and an HDR image, according to some embodiments. As shown in FIG. 2A, step 210 may involve a computing device 102 accessing a multi-channel HDR image 211 composed of pixels 212 (each denoted as "P"). As shown in FIG. 2A, the pixels 212 may be arranged according to a row / column layout, with the subscript "P" (e.g., "1,1") of each pixel 212 indicating the location of the pixel 212 according to row and column. In the example shown in FIG. 2A, the pixels 212 of the multi-channel image 108 are arranged in an equal number of rows and columns such that the multi-channel image 108 is a square image. However, it should be noted that the techniques described herein may be applied to a multi-channel image 108 having a different layout (e.g., an unequal row / column count). 2A, each pixel 212 may be composed of three subpixels 214: a red subpixel 214 (designated "R"), a green subpixel 214 (designated "G"), and a blue subpixel (designated "B") 214. However, it should be noted that each pixel 212 may be composed of any number of subpixels without departing from the scope of the present disclosure.

[0029] 2B illustrates step 220, which involves computing device 102 accessing multi-channel SDR image 221. As shown in FIG. 2B, multi-channel SDR image 221 is composed of pixels 222 (and sub-pixels 224) similar to pixels 212 (and sub-pixels 214) of multi-channel HDR image 211 shown in FIG. 2A. According to some embodiments, multi-channel SDR image 221 is a single-exposure capture of the same scene captured by multi-channel HDR image 211; thus, multi-channel SDR image 221 and multi-channel HDR image 211 are substantially related to one another. For example, if multi-channel HDR image 211 was generated using the EV−, EV0, and EV+ techniques described herein, multi-channel SDR image 221 can be based on the EV0 exposure (e.g., before the EV0 exposure is merged with the EV− and EV+ exposures to generate multi-channel HDR image 211). This approach can ensure that both multi-channel HDR image 211 and multi-channel SDR image 221 correspond to the same scene at the same moment in time. In this way, pixels in multi-channel HDR image 211 and multi-channel SDR image 221 can differ only in luminous intensity collected from the same point in the same scene (as opposed to differing in scene content due to motion resulting from the passage of time that occurs through successively captured exposures).

[0030] 2C illustrates step 230, which involves the computing device 102 generating a multi-channel gain map 231 (composed of pixels 232) by comparing the multi-channel HDR image 211 and the multi-channel SDR image 221 (shown as comparison 234 in FIG. 2C). Here, if it is desired to enable the multi-channel SDR image 221 to be reproduced using the multi-channel HDR image 211, a first approach may be utilized. In particular, the first approach involves dividing the value of each pixel of the multi-channel SDR image 221 by the value of the corresponding pixel of the multi-channel HDR image 211 to generate a quotient. Each quotient can then be assigned to the value of the corresponding pixel 232 in the multi-channel gain map 231. For example, the "P" of the multi-channel HDR image 211 may be 1,1 ” has a value of “5”, and the pixel indicated by “P 1,1 If the pixel indicated by "P" has a value of "1", the quotient will be "0.2" and the "P" of the multi-channel gain map 231 1,1 In this manner, and as will be explained in more detail herein, the pixel values ​​indicated by "P" of the multi-channel SDR image 221 are assigned to the pixel values ​​indicated by "P" of the multi-channel SDR image 221. 1,1 The pixels indicated by "P" are the pixels of the multi-channel HDR image 211. 1,1 ” in the multi-channel gain map 231. 1,1 In particular, the multiplication is performed by multiplying the pixel indicated by "P" (which has a value of "0.2") in the multi-channel SDR image 221. 1,1 2D , which produces a product "1" that corresponds to a value "1" for the pixel indicated by "." Storing the multi-channel gain map 231 along with the multi-channel HDR image 211 can therefore allow the multi-channel SDR image 221 to be reproduced independently of the multi-channel SDR image 221 itself. A more detailed description of various ways in which the multi-channel gain map 231 may be stored along with the corresponding multi-channel image is described below in connection with FIG. 2D .

[0031] Alternatively, if it is desired to enable the multi-channel HDR image 211 to be reproduced using the multi-channel SDR image 221, a second (different) approach may be utilized. In particular, the second approach involves dividing the value of each pixel of the multi-channel HDR image 211 by the value of the corresponding pixel of the multi-channel SDR image 221 to generate a quotient. Each quotient can then be assigned to the value of the corresponding pixel 232 in the multi-channel gain map 231. For example, 1,1 The pixel indicated by "P" in the multi-channel HDR image 211 has a value of "3". 1,1 If the pixel indicated by "P" has a value of "6", the quotient will be "2" and the "P" in the multi-channel gain map 231 1,1 In this way, and as will be explained in more detail herein, the value of the pixel indicated by "P" of the multi-channel HDR image 211 is assigned. 1,1 The pixels denoted by "P" are the pixels of the multi-channel SDR image 221. 1,1 ” in the multi-channel gain map 231. 1,1 In particular, the multiplication is performed by multiplying the pixels indicated by "P" (which have a value of "2") in the multi-channel SDR image 221. 1,1 2D , which corresponds to the value of the pixel indicated by "6." Storing the multi-channel gain map 231 with the multi-channel SDR image 221 can therefore allow the multi-channel HDR image 211 to be reproduced independently of the multi-channel HDR image 211 itself. Again, a more detailed description of various ways in which the multi-channel gain map 231 may be stored with the corresponding multi-channel image is described below in connection with FIG. 2D .

[0032] 2C (and described herein) constitutes a pixel-level comparison, it should be noted that embodiments are not so limited. Conversely, pixels of images may be compared to one another at any level of granularity without departing from the scope of this disclosure. For example, sub-pixels of multi-channel HDR image 211 and multi-channel SDR image 221 may be compared to one another (instead of, or in addition to, pixel-level comparison) such that multiple gain maps are generated under different comparison techniques (e.g., separate gain maps for each color channel).

[0033] Additionally, it should be noted that various optimizations may be employed when generating the gain map without departing from the scope of the present disclosure. For example, if two values ​​are identical to one another, a comparison operation may be skipped, and a single-bit value (e.g., “0”) may be assigned to the corresponding value in the gain map to minimize the size (i.e., storage requirements) of the gain map. Furthermore, the resolution of the gain map may be smaller than the resolution of the images being compared to generate the gain map. For example, to generate a gain map that is one-quarter the resolution of the first and second images, an approximation of every fourth pixel in a first image may be compared with a corresponding approximation of every fourth pixel in a second image. This approach substantially reduces the size of the gain map, but reduces the overall accuracy with which the first image can be reproduced from the second image and gain map (or vice versa). Furthermore, the first and second images may be resampled in any conceivable manner before generating the gain map. For example, the first and second images may undergo a local tone mapping operation before generating the gain map. In some embodiments, the first and second images used to generate the gain map may undergo a local tone mapping operation before generating the gain map. In some embodiments, a global tone map is generated and used in conjunction with the gain map to provide locally adaptive tone mapping. In some embodiments, the gain map is stored at multiple resolutions (or as a multi-scale image), and different gain map values ​​can be obtained from the gain map for use with different sized images that are displayed using the gain map.

[0034] 2D illustrates step 240, in accordance with some embodiments, in which the computing device 102 embeds a multi-channel gain map 231 into the multi-channel HDR image 211 or the multi-channel SDR image 221. In particular, when the first technique described above in connection with FIG. 2C is utilized, which enables the multi-channel HDR image 211 and the multi-channel gain map 231 to be used to reproduce the multi-channel SDR image 221, the computing device 102 embeds the multi-channel gain map 231 into the multi-channel HDR image 211 (thereby generating the enhanced multi-channel image 122). As illustrated in FIG. 2D , one technique for embedding the multi-channel gain map 231 into the multi-channel HDR image 211 includes interleaving each pixel 232 (of the multi-channel gain map 231) with respect to its corresponding pixel 212 (of the multi-channel HDR image 211). An alternative approach may involve embedding each pixel 232 (of the multi-channel gain map 231) into its corresponding pixel 212 (of the multi-channel gain map 231) as an additional channel of the pixel 212. Yet another approach may include embedding the multi-channel gain map 231 as metadata stored with the multi-channel HDR image 211. It should be noted that the foregoing approaches are exemplary and not intended to be limiting, and that the multi-channel gain map 231 (as well as other supplemental gain maps, if generated) may be stored with the multi-channel HDR image 211 using any conceivable approach without departing from the scope of the present disclosure.

[0035] 2E illustrates a method 250 for generating a gain map based on an SDR image and an HDR image, according to some embodiments. As shown in FIG. 2E, method 250 begins at step 252, where computing device 102 accesses an HDR image (e.g., as described above in connection with FIG. 2A). At step 254, computing device 102 accesses an SDR image (e.g., as described above in connection with FIG. 2B). At step 256, computing device 102 generates a gain map by comparing the HDR image with the SDR image, or vice versa (e.g., as described above in connection with FIG. 2C). At step 258, computing device 102 embeds the gain map into the HDR image or SDR image (thereby generating enhanced multi-channel image 122, e.g., as described above in connection with FIG. 2D).

[0036] 3A shows a diagram 300 of the computing device 102 of FIG. 1 with an additional computational module for generating a compressed enhanced multi-channel image 308 from the enhanced multi-channel image 122, the generation of which was described previously herein. The enhanced multi-channel image 122 includes a gain map 123 generated by comparing at least two multi-channel images 108 paired with one of the multi-channel images 108. In some embodiments, the compression module 302 processes the multi-channel images 108 (which may be in uncompressed form) individually or together with the gain map 123 to form the compressed multi-channel image 304. The compression module 302 also processes the gain map 123 (which may be in uncompressed form) individually or together with the multi-channel images 108 to form the compressed gain map 306. The compressed multi-channel image 304 may be combined with the compressed gain map 306 to form a compressed enhanced multi-channel image 308, which may be stored locally, for example, in the non-volatile memory 124 (or another local storage medium) of the computing device 102, and / or remotely, for example, in a cloud network-based service such as iCloud® managed by Apple®. In some embodiments, the remotely stored compressed enhanced multi-channel image 308 may be retrieved by a second computing device 102 and used to generate an uncompressed image having a dynamic range of luminance values ​​suitable for display by the second computing device 102.

[0037] 3B shows a diagram 310 for generating a compressed enhanced multi-channel image 324 by the computing device 102. The gain map generator 120 can compare the luminance values ​​of pixels in the first multi-channel image 108-A with the luminance values ​​of pixels in the second multi-channel image 108-B to generate the gain map 123. The gain map 123 can be defined based on the bit depth of each gain map value, e.g., at least 10 bits, and the gain map resolution, which may optionally include downsampling to reduce the storage requirements of the gain map 123. In some embodiments, the gain map 123 is reduced from full resolution, where each gain map value in the gain map 123 corresponds to a single pixel in each of the first and second multi-channel images 108-A, 108-B, to a lower resolution, where each gain value in the gain map 123 corresponds to multiple pixels in each of the first and second multi-channel images 108-A, 108-B. Reducing the resolution of the gain map 123 can affect image quality when the gain map 123 is used with an uncompressed version of the compressed multi-channel image 320 having a first dynamic range to generate a second image having a second dynamic range for display. Reducing the resolution of the gain map 123 typically reduces the contrast of images subsequently generated using the gain map 123. In some cases, a half or quarter resolution gain map 123 (the latter representing half the resolution in each dimension of the image) has limited impact on the image quality of images subsequently generated using the gain map 123. The gain map 123 provides a representation of the luminance difference between images having different dynamic ranges, for example, between an SDR image and an HDR image. In some cases, the first multi-channel image 108-A is an ideal (or reference) SDR image of the scene, and the second multi-channel image 108-B is an ideal (or reference) HDR image of the scene.Ideal SDR and HDR images can be generated manually by a user of the computing device 102, for example, using an image processing application, or automatically by an image capture application of the computing device 102. The gain map 123 is intended to enable storing only one image of a scene, such as an SDR image (or a version derived therefrom) or an HDR image (or a version derived therefrom), and subsequently generating a corresponding complementary image of the scene. For example, an SDR image can be stored with the gain map 123, and later, an HDR image can be generated by applying the gain map 123 to the SDR image. The gain map 123 allows for local tone mapping to be applied within a smaller area of ​​an image than a global tone map, which is applied to the entire image, depending on its resolution. Global tone mapping affects the entire image, and each pixel of the image is mapped using the same function for each pixel without considering the local context of nearby pixels. Local tone mapping affects local regions of the image, determining a mapping function that takes into account pixels neighboring each individual pixel, and can improve contrast between neighboring pixels compared to global tone mapping. In the embodiments described herein, the primary goal is to generate a base image derived from multiple (typically two) images, each optimized for a different dynamic range of luminance, along with a gain map 123 that captures the differences between the images. In some embodiments, the base image can be used to generate a first display image having a first dynamic range, e.g., an SDR display image, and the base image, along with the gain map, can be used to generate a second display image having a second dynamic range, e.g., an HDR display image. A full-resolution gain map 123 containing gain values ​​to use for each pixel in the associated base image can provide a high quality level, but requires a significant amount of storage.In some embodiments, the base image is an SDR image, and multiple gain maps 123 are generated, with each gain map 123 associated with a different HDR display capability. Storing multiple gain maps 123 along with a full-resolution SDR image may require more storage than desired (or available) by a user of the computing device 102. Reducing the resolution of the gain map 123 provides one form of storage reduction, but too aggressive a resolution reduction of the gain map 123 may result in artifacts when later regenerating an HDR image from the SDR (base) image and gain map 123 for display. Compression of the base image and gain map 123 (with possible modest resolution reduction of the gain map 123, such as half resolution corresponding to a gain map value per pair of pixels or quarter resolution corresponding to a gain map value per quad of pixels) can provide compact storage and high-quality results.

[0038] 3B , the gain map 123 generated by the gain map generator 120 from the first multi-channel image 108-A and the second multi-channel image 108-B is processed by the gain map compression module 312 to generate a compressed gain map 322. Separately, either the first multi-channel image 108-A or the second multi-channel image 108-B is selected by the image selection module 314, and the selected multi-channel image 316 is processed by the image compression module 318 to generate a compressed multi-channel image 320. The compressed multi-channel image 320 can be combined with the compressed gain map 322 to form an enhanced compressed multi-channel image 324, which can be stored locally on the computing device 102 or remotely on an external storage device, such as a cloud network-based service accessible to the computing device 102. The compressed multi-channel image 320 may later be decompressed (or uncompressed) to replicate the selected multi-channel image 316 (which may be the first multi-channel image 108-A or the second multi-channel image 108-B). The compressed gain map 322 may also be decompressed (or uncompressed) to recreate the gain map 123, which may be combined with an uncompressed version of the compressed multi-channel image 108 to generate a version of the first or second (i.e., unselected) multi-channel image 320-A, 108-B. In some embodiments, the image compression module 318 uses an image compression algorithm optimized for processing images, and the gain map compression module 312 uses a gain map compression algorithm optimized for processing the gain map 123, which may have substantially different characteristics than the images.

[0039] 3C shows a diagram 330 of another technique for generating a compressed enhanced multi-channel image 338. The gain map generator 120 processes the first multi-channel image 108-A and the second multi-channel image 108-B to generate a gain map 123, which is processed by a gain map compression module 312 to form a compressed gain map 306. Separately, the first multi-channel image 108-A and the second multi-channel image 108-B are processed together by an image compression module 332 to form a compressed multi-channel image 334. The compressed multi-channel image 334 and the compressed gain map 306 are combined to form the compressed multi-channel image 338. 3B , either the first multi-channel image 108-A or the second multi-channel image 108-B is selected and compressed to form the compressed multi-channel image 320, while in the technique of FIG. 3C , both the first and second multi-channel images 108-A, 108-B are processed together to generate the compressed multi-channel image 334. In some embodiments, the compressed multi-channel image 334 may be decompressed (or uncompressed) to form a version of either the first multi-channel image 108-A or the second multi-channel image 108-B. In some embodiments, the compressed multi-channel image 334 may be decompressed (or uncompressed) and combined with a decompressed version of the gain map 123 obtained from the compressed gain map 306 to form a corresponding complementary version of either the second multi-channel image 108-B or the first multi-channel image 108-A. The first and second multi-channel images 108-A, 108-B may have different dynamic ranges of luminance values, and the corresponding regenerated versions of the first and second multi-channel images 108-A, 108-B may also have different dynamic ranges of luminance values.

[0040] 3D shows a diagram 340 of a further technique for generating a compressed enhanced multi-channel image 344. A first multi-channel image 108-A (which may be an SDR image or an HDR image) may be processed by an image compression module 318 to generate a compressed first multi-channel image 342. A second multi-channel image 108-B (which may be a complementary HDR image or a complementary SDR image) may be processed together with the compressed first multi-channel image 342 by a gain map generator 120 to generate a gain map 123. The gain map 123 may then be processed by a gain map compression module 312 to form a compressed gain map 322, which may be combined with the compressed first multi-channel image 342 to form a compressed enhanced multi-channel image 344, which may be stored locally on the computing device 102 or remotely in an accessible storage facility separate from the computing device 102, such as a cloud network-based server. The compressed enhanced multi-channel image 344 may be retrieved from local or remote storage by the computing device 102 (or possibly by another computing device 102) and used to recreate versions of the first and second multi-channel images 108-A, 108-B. The compressed first multi-channel image 342 may be extracted from the compressed enhanced multi-channel image 344 and decompressed (or uncompressed) to replicate a version of the first multi-channel image 108-A. The compressed gain map 322 may be extracted from the compressed enhanced multi-channel image 344 and decompressed (or uncompressed) to obtain a version of the gain map 123, which may be combined with a version of the first multi-channel image 108-A to generate a version of the second multi-channel image 108-B suitable for display.

[0041] 3B-3D , in which the selected multi-channel image 316 and the gain map 123 are independently compressed, may be less than ideal because, in some cases, the selected multi-channel image 316 and the gain map 123 may be highly correlated with each other. Independent compression and subsequent decompression (or uncompression) of the compressed multi-channel image 320 and the compressed gain map 322, followed by application of the decompressed gain map to the decompressed multi-channel image, may introduce compression artifacts into the resulting image. For example, a compression artifact affecting a gain map pixel (or set of pixels) and a separate compression artifact affecting an image pixel (or set of pixels) may introduce substantial errors when decompressing the gain map and applying the gain map 123 to the decompressed image. An improved implementation of compression may include joint (or closed-loop) compression, which uses a combination of the image and the gain map 123 to generate a compressed version of the image included in the compressed enhanced multi-channel image.

[0042] 3E shows another example diagram 350 of generating a compressed enhanced multi-channel image 358. A combined (joint) gain map generation and compression module 352 processes the first multi-channel image 108-A and the second multi-channel image 108-B together to form the compressed enhanced multi-channel image 358, which includes a compressed multi-channel image 354 and a compressed gain map 356. The first and second multi-channel images 108-A, 108-B may each have a different dynamic range of luminance values; for example, the first multi-channel image 108-A may be an SDR multi-channel image and the second multi-channel image 108-B may be an HDR multi-channel image. In some embodiments, one version of the first and second multi-channel images 108-A, 108-B may be generated using the compressed multi-channel image 354, while the other version of the first and second multi-channel images 108-A, 108-B may be generated using the compressed multi-channel image 354 in combination with the compressed gain map 356.

[0043] In some embodiments, additional metadata is generated and stored with the compressed enhanced multi-channel images 308, 324, 338, 344, 358. Exemplary metadata include content information such as whether the image contains a human face, the maximum amount of headroom available for processing the image content, an offset value for the gain map 123, and / or an error map associated with compression artifacts in the compressed gain map.

[0044] 4 shows an example diagram 400 of generating an HDR multi-channel image 426 targeted for display by a computing device 102 from a compressed enhanced multi-channel image 406. The compressed enhanced multi-channel image 406 may be pre-generated by the computing device 102, which decompresses and generates the HDR multi-channel image 426 targeted for display, or by a separate computing device 102. For example, the compressed multi-channel image 406 may be generated on a first computing device 102, stored on a cloud network-based server, retrieved by a second computing device 102, and processed by the second computing device 102 for presentation on a display associated with the second computing device 102. The HDR multi-channel image 426 generated for display by the second computing device 102 may be processed according to known characteristics of the display, which may not be known at the time the compressed enhanced multi-channel image 406 is generated by the first computing device 102.

[0045] The computing device 102 can extract the compressed multi-channel image 402 from the compressed enhanced multi-channel image 406 and decompress the extracted compressed enhanced multi-channel image 406 to generate a multi-channel image base layer 408, which may be in SDR format in some embodiments. The computing device 102 can also extract the compressed gain map 404 from the compressed enhanced multi-channel image 406 and decompress the extracted compressed gain map 404 to generate an uncompressed version of the gain map 410. The gain map 410 can be processed by a renormalization module 414, which considers minimum and maximum logarithmic (log2) values ​​418 when processing for different color channels. In some cases, the gain map values ​​for the red channel are scaled and processed in the logarithmic domain, including via an exponential function module 416, and the gain map values ​​for the cyan color channel are scaled and processed in the linear domain. The values ​​of the gain map 410 may be appropriately scaled in a gain map scaling module 428 using knowledge of the peak values ​​430 of the display on which the final HDR multi-channel image 426 is intended to be displayed. The scaled gain map values ​​for the color channels (after passing through the applicable degamma function module 412) may be applied to the multi-channel image base layer 408 in a gain mapping module 422, which also uses offset values ​​420 previously stored as metadata with the compressed gain map 404. The output of the gain mapping module 422 is further processed by a color management module 424 to generate an HDR multi-channel image optimized for the particular display. In some embodiments, metadata such as the offset values ​​420 and minimum and maximum log2 values ​​418 are stored with the gain map 410 (and compressed with the gain map 410) or with the compressed gain map 404.In some embodiments, gain map 410 uses normalized values ​​with a range of valid values ​​from 0 to 1, and the re-normalized version of gain map 410 includes the full range of gain map values ​​that were originally calculated when determining the original version of gain map 410 (when comparing the original SDR image and HDR image). In some embodiments, the re-normalized gain map values ​​are log2 scaled values, and the linear versions of the re-normalized gain map values ​​are exponential values, e.g., a log2 scaled value x is a linear scaled value 2. x In some embodiments, the scaling of a portion of the gain map 410 by the gain map scaling module 428 is performed in the logarithmic domain (e.g., for certain color channels). In some embodiments, the scaling of a portion of the gain map 410 by the gain map scaling module 428 is performed in the linear domain (e.g., for certain other color channels). In some embodiments, the amount of scaling applied to generate scaled gain map values ​​to apply to the base layer image (after the degamma module 412) is based on the capabilities of the target display, display environmental conditions (e.g., brighter or dimmer ambient light), and / or other metadata values. In some embodiments, the gain map 410 is generated at the source computing device 102 by calculating a ratio of pixel luminance values ​​and adding an offset value 420 to divisor pixel luminance values ​​that are zero to ensure that division by zero is not performed in the ratio calculation. The offset value 420 may then be removed when applying the regenerated gain map (in the gain mapping module 422). In some embodiments, the offset value 420 may be based on the original SDR image, the original HDR image, and / or may be selected based on compression and / or gain mapping considerations. In some embodiments, the offset value 420 is selected to optimize the gain map storage.

[0046] In some embodiments, the computing device 102 determines the gain map 123, compresses the gain map 123 to form a compressed gain map 306, 322, 356, decompresses the compressed gain map 306, 322, 356 to form an uncompressed version of the gain map 123, compares the values ​​of the (original) gain map 123 with the values ​​of the uncompressed version of the gain map 123, and determines an error map that captures the error from the compression of the gain map 123. The computing device 102 may store the error map together with the compressed enhanced multi-channel image 308, 324, 338, 344, 358 (either together with the compressed gain map 306, 322, 356 or separately with accompanying metadata). In some embodiments, the computing device 102 determines multiple gain maps 123, each intended for a different use, such as for a different target display having different characteristics, e.g., size, resolution, color gamut coverage, maximum brightness, or for later presentation of an image derived from the base image and gain map, each image having different style characteristics, i.e., different versions of the image. For example, the computing device 102 may generate multiple gain maps 123 for different peak display values ​​of 500 nits, 1000 nits, 2000 nits, and 4000 nits, and use the multiple gain maps 123 to generate different images optimized for different displays having different peak display values. In some embodiments, the computing device 102 generates a gain map 123 for the multi-channel image 108 and then transcodes the multi-channel image 108 into another image format different from the image format used for the original multi-channel image 108, such as when the color space used for the image has changed. The computing device 102 can recalculate the gain map for the transcoded multi-channel image 108 from the original gain map 123 or from a newly calculated gain map based on the transcoded multi-channel image 108.

[0047] In some embodiments, the computing device 102 determines the gain maps 123 at multiple resolutions, e.g., the multi-scale gain maps can be compressed and stored along with the base image, which may also be at multiple resolutions or resampled to multiple resolutions, and an appropriate gain map can be derived from the multi-scale gain map and applied to the base image (or a resampled version of the base image) to provide an image of the relevant resolution for display. Thus, the multi-scale gain map can be used to apply locally adaptive tone mapping to images sized for different output displays. In some embodiments, a global tone map is applied to the baseline image before (or after) applying the gain map to obtain the image for display, and the gain map provides adaptive local tone mapping that is separate from the global tone mapping.

[0048] 5A shows a flowchart 500 of an exemplary method for image management by a computing device 102. At 502, the computing device generates a compressed version of the image and a compressed version of the gain map from a standard dynamic range (SDR) image of a scene and a high dynamic range (HDR) image of the scene. At 504, the computing device 102 combines the compressed version of the image with the compressed version of the gain map to form a compressed enhanced image. At 506, the computing device 102 stores the compressed enhanced image in a non-volatile storage medium.

[0049] 5B shows a flowchart 520 of another exemplary method for image management by a second computing device 102. At 522, the second computing device 102 obtains a compressed enhanced image. At 524, the second computing device 102 extracts a compressed version of the image and a compressed version of the gain map from the compressed enhanced image. At 526, the second computing device generates an uncompressed version of the image from the compressed version of the image. At 528, the second computing device generates an uncompressed version of the gain map from the compressed version of the gain map. At 530, the second computing device applies the uncompressed version of the gain map to the uncompressed version of the image to generate a second image formatted for display by the second computing device 102, the second image and the uncompressed version of the image having different dynamic ranges of brightness values.

[0050] In some embodiments, the compressed image includes a compressed version of the SDR image. In some embodiments, the second image includes a version of the HDR image. In some embodiments, the compressed image includes a compressed version of the HDR image. In some embodiments, the second image includes a version of the SDR image. In some embodiments, the method performed by the computing device 102 further includes the computing device 102 i) generating a gain map by comparing luminance values ​​of pixels in the HDR image with luminance values ​​of corresponding pixels in the SDR image, ii) processing the SDR image or the HDR image with an image compression module to generate a compressed version of the image, and iii) processing the gain map with a gain compression module to generate a compressed version of the gain map. In some embodiments, the method performed by the computing device 102 further includes the computing device 102 i) generating a gain map by comparing luminance values ​​of pixels in the HDR image with luminance values ​​of corresponding pixels in the SDR image, ii) processing the SDR image and the HDR image together with the image compression module to generate a compressed version of the image, and iii) processing the gain map with the gain compression module to generate a compressed version of the gain map. In some embodiments, the method performed by the computing device 102 further includes the computing device 102: i) generating a compressed version of the image by processing the SDR image with an image compression module; ii) generating a gain map by comparing luminance values ​​of pixels in the HDR image with luminance values ​​of corresponding pixels in the compressed version of the image; and iii) generating a compressed version of the gain map by processing the gain map with the gain compression module.In some embodiments, the computing device 102 generates a compressed version of the image and a compressed version of the gain map by jointly generating a compressed version of the gain map and a compressed version of the image from the SDR image and the HDR image using a combined gain map generation and compression module. In some embodiments, the compressed version of the gain map is derived from a gain map having a linear resolution in each of two dimensions that is identical to the linear resolution of the SDR image and the HDR image. In some embodiments, the compressed version of the gain map is derived from a gain map having a linear resolution in at least one dimension that is smaller than the corresponding linear resolution of the SDR image and the HDR image. In some embodiments, the compressed version of the gain map is generated using a first compression scheme optimized for the gain map, and the compressed version of the image is generated using a second compression scheme optimized for the image.

[0051] FIG. 6 shows a detailed diagram of a computing device 600 that can be used to perform the various techniques described herein, according to some embodiments. In particular, the detailed diagram illustrates various components that may be included in the computing device 102 described in connection with FIG. 1. As shown in FIG. 6, the computing device 600 may include a processor 602, which represents a microprocessor or controller for controlling the overall operation of the computing device 600. The computing device 600 may also include a user input device 608 that allows a user of the computing device 600 to interact with the computing device 600. For example, the user input device 608 may take various forms, such as input in the form of buttons, keypads, dials, a touchscreen, an audio input interface, a visual / image capture input interface, sensor data, etc. Additionally, the computing device 600 may include a display 610 that may be controlled by the processor 602 (e.g., via a graphics component) to display information to the user. A data bus 616 may facilitate data transfer between at least the storage device 640, the processor 602, and the controller 613. The controller 613 may be used to interface with and control different devices through an device control bus 614. The computing device 600 may also include a network / bus interface 611 that couples to a data link 612. In the case of a wireless connection, the network / bus interface 611 may include a wireless transceiver.

[0052] As mentioned above, the computing device 600 also includes a storage device 640, which may include a single disk or a collection of disks (e.g., a hard drive). In some embodiments, the storage device 640 may include flash memory, semiconductor (solid-state) memory, or the like. The computing device 600 may also include a random access memory (RAM) 620 and a read-only memory (ROM) 622. The ROM 622 may store executed programs, utilities, or processes in a non-volatile manner. The RAM 620 may provide volatile data storage and stores instructions related to the operation of applications executing on the computing device 600, e.g., the image analyzer 110 / gain map generator 120.

[0053]

[0006] Techniques described herein include techniques for image management. According to some embodiments, a first technique can be implemented by a computing device and includes: (1) generating a compressed version of the image and a compressed version of the gain map from a standard dynamic range (SDR) image of a scene and a high dynamic range (HDR) image of the scene; (2) combining the compressed version of the image with the compressed version of the gain map to form a compressed enhanced image; and (3) storing the compressed enhanced image in a non-volatile storage medium.

[0054] According to some embodiments, the aforementioned techniques may further include, by a second computing device, (1) obtaining a compressed enhanced image; (2) extracting from the compressed enhanced image a compressed version of the image and a compressed version of the gain map; (3) generating an uncompressed version of the image from the compressed version of the image; (4) generating an uncompressed version of the gain map from the compressed version of the gain map; and (5) applying the uncompressed version of the gain map to the uncompressed version of the image to generate a second image formatted for display by the second computing device, wherein the second image and the uncompressed version of the image have different dynamic ranges of brightness values.

[0055] According to some embodiments, the compressed version of the image comprises a compressed version of an SDR image and the second image comprises a version of an HDR image. According to some embodiments, the compressed version of the image comprises a compressed version of an HDR image and the second image comprises a version of an SDR image. According to some embodiments, the compressed version of the gain map is generated using a lossy compression module selected for use with the gain map compression. According to some embodiments, the compressed version of the image is generated using a lossy compression module selected for use with the image compression.

[0056] According to some embodiments, generating a compressed version of the image and a compressed version of the gain map includes (1) generating a gain map by comparing luminance values ​​of pixels in the HDR image with luminance values ​​of corresponding pixels in the SDR image, (2) generating a compressed version of the image by processing the SDR image or the HDR image with an image compression module, and (3) generating a compressed version of the gain map by processing the gain map with a gain compression module.

[0057] According to some embodiments, generating a compressed version of the image and a compressed version of the gain map includes (1) generating a gain map by comparing luminance values ​​of pixels in the HDR image with luminance values ​​of corresponding pixels in the SDR image, (2) generating a compressed version of the image by processing the SDR image and the HDR image together using an image compression module, and (3) generating a compressed version of the gain map by processing the gain map using a gain compression module.

[0058] According to some embodiments, generating a compressed version of the image and a compressed version of the gain map includes (1) generating a compressed version of the image by processing the SDR image with an image compression module; (2) generating a gain map by comparing luminance values ​​of pixels in the HDR image with luminance values ​​of corresponding pixels in the compressed version of the image; and 3) generating a compressed version of the gain map by processing the gain map with a gain compression module.

[0059] According to some embodiments, generating a compressed version of the image and a compressed version of the gain map includes jointly generating a compressed version of the gain map and a compressed version of the image from the SDR image and the HDR image using a combined gain map generation and compression module.

[0060] According to some embodiments, the compressed version of the gain map is derived from a gain map having a linear resolution in each of two dimensions that is identical to the linear resolution of the SDR image and the HDR image. According to some embodiments, the compressed version of the gain map is derived from a gain map having a linear resolution in at least one dimension that is smaller than the corresponding linear resolution of the SDR image and the HDR image. According to some embodiments, the compressed version of the gain map is generated using a first compression scheme optimized for the gain map, and the compressed version of the image is generated using a second compression scheme optimized for the image.

[0061] According to some embodiments, an offset value based on pixel values ​​of the SDR image and / or pixel values ​​of the HDR image is used by the computing device when generating the gain map. According to some embodiments, the offset value is selected by the computing device to optimize storage of a compressed version of the gain map.

[0062] According to some embodiments, the foregoing techniques may further include, by the computing device, (1) generating an uncompressed version of the gain map from a compressed version of the gain map, (2) determining an error map based on comparing the uncompressed version of the gain map with the original gain map used to generate the compressed version of the gain map, and (3) storing the compressed version of the error map with the compressed enhanced image. According to some embodiments, the compressed version of the error map is compressed using a lossless compression module and the compressed version of the gain map is compressed using a lossy compression module.

[0063] Various aspects, embodiments, implementations, or features of the described embodiments can be used individually or in any combination. Various aspects of the described embodiments can be implemented by software, hardware, or a combination of hardware and software. The described embodiments can also be embodied as computer-readable code on a computer-readable medium. The computer-readable medium is any data storage device that can store data which can thereafter be read by a computer system. Examples of computer-readable media include read-only memory, random-access memory, CD-ROMs, DVDs, magnetic tape, hard disk drives, solid-state drives, and optical data storage devices. The computer-readable medium can also be distributed over network-coupled computer systems so that the computer-readable code is stored and executed in a distributed fashion.

[0064] In the foregoing description, for purposes of explanation, specific terminology was used to provide a thorough understanding of the described embodiments. However, it will be apparent to those skilled in the art that specific details are not required to practice the described embodiments. Thus, the descriptions of the specific embodiments set forth above are presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the described embodiments to the precise forms disclosed. It will be apparent to those skilled in the art that numerous modifications and variations are possible in light of the above teachings.

Claims

1. 1. A method for image management, the method comprising, in a computing device, generating a compressed version of the image and a compressed version of the gain map from a standard dynamic range (SDR) image of a scene and a high dynamic range (HDR) image of the scene; combining the compressed version of the image with the compressed version of the gain map to form a compressed enhanced image; storing the compressed enhanced image in a non-volatile storage medium; A method comprising:

2. In a second computing device, obtaining the compressed enhanced image; extracting from the compressed enhanced image the compressed version of the image and the compressed version of the gain map; generating an uncompressed version of the image from the compressed version of the image; generating an uncompressed version of the gain map from the compressed version of the gain map; applying the uncompressed version of the gain map to the uncompressed version of the image to generate a second image formatted for display by the second computing device; The method of claim 1 , wherein the second image and the uncompressed version of the image have different dynamic ranges of luminance values.

3. the compressed version of the image comprises a compressed version of the SDR image; the second image includes a version of the HDR image; The method of claim 2.

4. the compressed version of the image includes a compressed version of the HDR image; the second image comprises a version of the SDR image; The method of claim 2.

5. The method of claim 1 , wherein the compressed version of the gain map is generated using a lossy compression module selected for use with gain map compression.

6. The method of claim 1 , wherein the compressed version of the image is generated using a lossy compression module selected for use with image compression.

7. generating the compressed version of the image and the compressed version of the gain map generating a gain map by comparing luminance values ​​of pixels in the HDR image with luminance values ​​of corresponding pixels in the SDR image; generating the compressed version of the image by processing the SDR or HDR image with an image compression module; and generating the compressed version of the gain map by processing the gain map with a gain compression module.

8. generating the compressed version of the image and the compressed version of the gain map generating a gain map by comparing luminance values ​​of pixels in the HDR image with luminance values ​​of corresponding pixels in the SDR image; generating the compressed version of the image by jointly processing the SDR image and the HDR image with an image compression module; and generating the compressed version of the gain map by processing the gain map with a gain compression module.

9. generating the compressed version of the image and the compressed version of the gain map generating the compressed version of the image by processing the SDR image with an image compression module; generating a gain map by comparing luminance values ​​of pixels in the HDR image with luminance values ​​of corresponding pixels in the compressed version of the image; and generating the compressed version of the gain map by processing the gain map with a gain compression module.

10. generating the compressed version of the image and the compressed version of the gain map 2. The method of claim 1, comprising using a combined gain map generation and compression module to jointly generate the compressed version of the gain map and the compressed version of the image from the SDR image and the HDR image.

11. The method of claim 1 , wherein the compressed version of the gain map is derived from a gain map having a linear resolution in each of two dimensions that is the same as the linear resolution of the SDR image and the HDR image.

12. The method of claim 1 , wherein the compressed version of the gain map is derived from a gain map having a linear resolution in at least one dimension that is less than the corresponding linear resolutions of the SDR image and the HDR image.

13. the compressed version of the gain map is generated using a first compression scheme optimized for gain maps; the compressed version of the image is generated using a second compression scheme optimized for the image. The method of claim 1.

14. The method of claim 1 , wherein an offset value based on pixel values ​​of the SDR image and / or pixel values ​​of the HDR image is used by the computing device when generating the gain map.

15. The method of claim 14 , wherein the offset value is selected by the computing device to optimize storage of the compressed version of the gain map.

16. the computing device: generating an uncompressed version of the gain map from the compressed version of the gain map; determining an error map based on comparing the uncompressed version of the gain map to an original gain map used to generate the compressed version of the gain map; The method of claim 1 , further comprising: storing a compressed version of the error map together with the compressed enhanced image.

17. the compressed version of the error map is compressed using a lossless compression module; the compressed version of the gain map is compressed using a lossy compression module.

17. The method of claim 16.

18. 1. A non-transitory computer-readable storage medium configured to store instructions that, when executed by at least one processor included in a computing device, cause the computing device to: generating a compressed version of the image and a compressed version of the gain map from a standard dynamic range (SDR) image of a scene and a high dynamic range (HDR) image of the scene; combining the compressed version of the image with the compressed version of the gain map to form a compressed enhanced image; and storing the compressed enhanced image in a non-volatile storage medium.

19. The steps include, on a second computing device: obtaining the compressed enhanced image; extracting from the compressed enhanced image the compressed version of the image and the compressed version of the gain map; generating an uncompressed version of the image from the compressed version of the image; generating an uncompressed version of the gain map from the compressed version of the gain map; applying the uncompressed version of the gain map to the uncompressed version of the image to generate a second image formatted for display by the second computing device; 20. The non-transitory computer-readable storage medium of claim 18, wherein the second image and the uncompressed version of the image have different dynamic ranges of luminance values.

20. the compressed version of the image comprises a compressed version of the SDR image; the second image includes a version of the HDR image; 20. The non-transitory computer-readable storage medium of claim 19.

21. the compressed version of the image includes a compressed version of the HDR image; the second image comprises a version of the SDR image; 20. The non-transitory computer-readable storage medium of claim 19.

22. 20. The non-transitory computer-readable storage medium of claim 18, wherein the compressed version of the gain map is generated using a lossy compression module selected for use with gain map compression.

23. 20. The non-transitory computer-readable storage medium of claim 18, wherein the compressed version of the image is generated using a lossy compression module selected for use with image compression.

24. generating the compressed version of the image and the compressed version of the gain map generating a gain map by comparing luminance values ​​of pixels in the HDR image with luminance values ​​of corresponding pixels in the SDR image; generating the compressed version of the image by processing the SDR or HDR image with an image compression module; and generating the compressed version of the gain map by processing the gain map with a gain compression module.

25. generating the compressed version of the image and the compressed version of the gain map generating a gain map by comparing luminance values ​​of pixels in the HDR image with luminance values ​​of corresponding pixels in the SDR image; generating the compressed version of the image by jointly processing the SDR image and the HDR image with an image compression module; and generating the compressed version of the gain map by processing the gain map with a gain compression module.

26. generating the compressed version of the image and the compressed version of the gain map generating the compressed version of the image by processing the SDR image with an image compression module; generating a gain map by comparing luminance values ​​of pixels in the HDR image with luminance values ​​of corresponding pixels in the compressed version of the image; and generating the compressed version of the gain map by processing the gain map with a gain compression module.

27. generating the compressed version of the image and the compressed version of the gain map 20. The non-transitory computer-readable storage medium of claim 18, comprising using a combined gain map generation and compression module to jointly generate the compressed version of the gain map and the compressed version of the image from the SDR image and the HDR image.

28. 20. The non-transitory computer-readable storage medium of claim 18, wherein the compressed version of the gain map is derived from a gain map having a linear resolution in each of two dimensions that is the same as the linear resolution of the SDR image and the HDR image.

29. 20. The non-transitory computer-readable storage medium of claim 18, wherein the compressed version of the gain map is derived from a gain map having a linear resolution in at least one dimension that is less than a corresponding linear resolution of the SDR image and the HDR image.

30. the compressed version of the gain map is generated using a first compression scheme optimized for gain maps; the compressed version of the image is generated using a second compression scheme optimized for the image.

20. The non-transitory computer-readable storage medium of claim 18.

31. 20. The non-transitory computer-readable storage medium of claim 18, wherein an offset value based on pixel values ​​of the SDR image and / or pixel values ​​of the HDR image is used by the computing device when generating the gain map.

32. 32. The non-transitory computer-readable storage medium of claim 31, wherein the offset value is selected by the computing device to optimize storage of the compressed version of the gain map.

33. The steps include the step of: generating an uncompressed version of the gain map from the compressed version of the gain map; determining an error map based on comparing the uncompressed version of the gain map to an original gain map used to generate the compressed version of the gain map; 20. The non-transitory computer-readable storage medium of claim 18, further comprising: storing a compressed version of the error map with the compressed enhanced image.

34. the compressed version of the error map is compressed using a lossless compression module; the compressed version of the gain map is compressed using a lossy compression module.

34. The non-transitory computer-readable storage medium of claim 33.

35. 1. A computing device configured to manage images, the computing device comprising: at least one processor; at least one memory that stores instructions that, when executed by the at least one processor, cause the computing device to: generating a compressed version of the image and a compressed version of the gain map from a standard dynamic range (SDR) image of a scene and a high dynamic range (HDR) image of the scene; combining the compressed version of the image with the compressed version of the gain map to form a compressed enhanced image; storing the compressed enhanced image in a non-volatile storage medium.

36. The steps include, on a second computing device: obtaining the compressed enhanced image; extracting from the compressed enhanced image the compressed version of the image and the compressed version of the gain map; generating an uncompressed version of the image from the compressed version of the image; generating an uncompressed version of the gain map from the compressed version of the gain map; applying the uncompressed version of the gain map to the uncompressed version of the image to generate a second image formatted for display by the second computing device; 36. The computing device of claim 35, wherein the second image and the uncompressed version of the image have different dynamic ranges of luminance values.

37. the compressed version of the image comprises a compressed version of the SDR image; the second image includes a version of the HDR image; 37. The computing device of claim 36.

38. the compressed version of the image includes a compressed version of the HDR image; the second image comprises a version of the SDR image; 37. The computing device of claim 36.

39. 36. The computing device of claim 35, wherein the compressed version of the gain map is generated using a lossy compression module selected for use with gain map compression.

40. 36. The computing device of claim 35, wherein the compressed version of the image is generated using a lossy compression module selected for use with image compression.

41. generating the compressed version of the image and the compressed version of the gain map generating a gain map by comparing luminance values ​​of pixels in the HDR image with luminance values ​​of corresponding pixels in the SDR image; generating the compressed version of the image by processing the SDR or HDR image with an image compression module; and generating the compressed version of the gain map by processing the gain map with a gain compression module.

42. generating the compressed version of the image and the compressed version of the gain map generating a gain map by comparing luminance values ​​of pixels in the HDR image with luminance values ​​of corresponding pixels in the SDR image; generating the compressed version of the image by jointly processing the SDR image and the HDR image with an image compression module; and generating the compressed version of the gain map by processing the gain map with a gain compression module.

43. generating the compressed version of the image and the compressed version of the gain map generating the compressed version of the image by processing the SDR image with an image compression module; generating a gain map by comparing luminance values ​​of pixels in the HDR image with luminance values ​​of corresponding pixels in the compressed version of the image; and generating the compressed version of the gain map by processing the gain map with a gain compression module.

44. generating the compressed version of the image and the compressed version of the gain map 36. The computing device of claim 35, further comprising using a combined gain map generation and compression module to jointly generate the compressed version of the gain map and the compressed version of the image from the SDR image and the HDR image.

45. 36. The computing device of claim 35, wherein the compressed version of the gain map is derived from a gain map having a linear resolution in each of two dimensions that is the same as the linear resolution of the SDR image and the HDR image.

46. 36. The computing device of claim 35, wherein the compressed version of the gain map is derived from a gain map having a linear resolution in at least one dimension that is less than a corresponding linear resolution of the SDR image and the HDR image.

47. the compressed version of the gain map is generated using a first compression scheme optimized for gain maps; the compressed version of the image is generated using a second compression scheme optimized for the image.

36. The computing device of claim 35.

48. 36. The computing device of claim 35, wherein an offset value based on pixel values ​​of the SDR image and / or pixel values ​​of the HDR image is used by the computing device when generating the gain map.

49. 49. The computing device of claim 48, wherein the offset value is selected by the computing device to optimize storage of the compressed version of the gain map.

50. The steps include the step of: generating an uncompressed version of the gain map from the compressed version of the gain map; determining an error map based on comparing the uncompressed version of the gain map to an original gain map used to generate the compressed version of the gain map; 36. The computing device of claim 35, further comprising: storing a compressed version of the error map with the compressed enhanced image.

51. the compressed version of the error map is compressed using a lossless compression module; the compressed version of the gain map is compressed using a lossy compression module.

51. The computing device of claim 50.

52. 1. A computing device configured to manage images, the computing device comprising: means for generating a compressed version of the image and a compressed version of the gain map from a standard dynamic range (SDR) image of a scene and a high dynamic range (HDR) image of said scene; means for combining the compressed version of the image with the compressed version of the gain map to form a compressed enhanced image; means for storing the compressed enhanced image in a non-volatile storage medium.

53. The second computing device means for obtaining the compressed enhanced image; means for extracting from the compressed enhanced image the compressed version of the image and the compressed version of the gain map; means for generating an uncompressed version of the image from the compressed version of the image; means for generating an uncompressed version of the gain map from the compressed version of the gain map; means for applying the uncompressed version of the gain map to the uncompressed version of the image to generate a second image formatted for display by the second computing device; 53. The computing device of claim 52, wherein the second image and the uncompressed version of the image have different dynamic ranges of luminance values.

54. the compressed version of the image comprises a compressed version of the SDR image; the second image includes a version of the HDR image; 54. The computing device of claim 53.

55. the compressed version of the image includes a compressed version of the HDR image; the second image comprises a version of the SDR image; 54. The computing device of claim 53.

56. 53. The computing device of claim 52, wherein the compressed version of the gain map is generated using a lossy compression module selected for use with gain map compression.

57. 53. The computing device of claim 52, wherein the compressed version of the image is generated using a lossy compression module selected for use with image compression.

58. generating the compressed version of the image and the compressed version of the gain map generating a gain map by comparing luminance values ​​of pixels in the HDR image with luminance values ​​of corresponding pixels in the SDR image; generating the compressed version of the image by processing the SDR or HDR image with an image compression module; and generating the compressed version of the gain map by processing the gain map with a gain compression module.

59. generating the compressed version of the image and the compressed version of the gain map generating a gain map by comparing luminance values ​​of pixels in the HDR image with luminance values ​​of corresponding pixels in the SDR image; generating the compressed version of the image by jointly processing the SDR image and the HDR image with an image compression module; and generating the compressed version of the gain map by processing the gain map with a gain compression module.

60. generating the compressed version of the image and the compressed version of the gain map generating the compressed version of the image by processing the SDR image with an image compression module; generating a gain map by comparing luminance values ​​of pixels in the HDR image with luminance values ​​of corresponding pixels in the compressed version of the image; and generating the compressed version of the gain map by processing the gain map with a gain compression module.

61. generating the compressed version of the image and the compressed version of the gain map 53. The computing device of claim 52, comprising using a combined gain map generation and compression module to jointly generate the compressed version of the gain map and the compressed version of the image from the SDR image and the HDR image.

62. 53. The computing device of claim 52, wherein the compressed version of the gain map is derived from a gain map having a linear resolution in each of two dimensions that is the same as the linear resolution of the SDR image and the HDR image.

63. 53. The computing device of claim 52, wherein the compressed version of the gain map is derived from a gain map having a linear resolution in at least one dimension that is less than a corresponding linear resolution of the SDR image and the HDR image.

64. the compressed version of the gain map is generated using a first compression scheme optimized for gain maps; the compressed version of the image is generated using a second compression scheme optimized for the image.

53. The computing device of claim 52.

65. 53. The computing device of claim 52, wherein an offset value based on pixel values ​​of the SDR image and / or pixel values ​​of the HDR image is used by the computing device when generating the gain map.

66. 66. The computing device of claim 65, wherein the offset value is selected by the computing device to optimize storage of the compressed version of the gain map.

67. means for generating an uncompressed version of the gain map from the compressed version of the gain map; means for determining an error map based on comparing the uncompressed version of the gain map to an original gain map used to generate the compressed version of the gain map; means for storing a compressed version of the error map together with the compressed enhanced image; 53. The computing device of claim 52, further comprising:

68. the compressed version of the error map is compressed using a lossless compression module; the compressed version of the gain map is compressed using a lossy compression module.

68. The computing device of claim 67.

Images