Techniques for pre-processing images to improve gain mapping compression results

Abstract

Techniques for pre-processing an image to improve gain map compression results are described. The gain map may be generated by comparing the first image to the second image, and then compressed to form a compressed gain map. The compressed gain map may be combined with a compressed version of the first image to form a compressed enhanced image. The compressed enhanced image may later be decompressed to generate an uncompressed version of the first image, and the gain map is applied to the uncompressed version of the first image to generate a version of the second image, providing rendering on a target display. The compressed version of the first image and a compressed gain map may be generated separately by an image compression module and a gain map compression module, or jointly by a combined gain map generation and compression module.

Description

Technical Field

[0001] Embodiments described herein set forth techniques for preprocessing images to improve gain map compression results. A gain map can be generated by comparing a first image to a second image, and then compressed to form a compressed gain map. The compressed gain map can be combined with a compressed version of the first image to form a compressed enhanced image. The compressed enhanced image can later be decompressed and the gain map applied to the first image to generate a version of the second image, thereby providing reproduction on a target display. Background Art

[0002] The dynamic range of an image refers to the range of pixel values ​​(also called "brightness") between the brightest and darkest parts of an image. It is worth noting that, relative to human visual perception of a scene, an image sensor captures a limited range of brightness in a single exposure of that scene. Images with a limited range are referred to herein as standard dynamic range (SDR) images.

[0003] Despite image sensor limitations, improvements in computational photography allow a larger range of luminance values ​​to be captured by an image sensor using multiple images that are processed together to form an image with a wider range (referred to herein as a high dynamic range (HDR) image). An HDR image can be formed by (1) capturing multiple bracketed images, i.e., separate SDR images that are each captured using a different exposure value (also referred to as an "aperture"), and (2) merging the bracketed SDR images into a single HDR image that combines aspects from the different exposures. The single HDR image includes a wider dynamic range of luminance values ​​compared to the narrower range of luminance values ​​in each of the separate SDR images. HDR images can be considered superior to SDR images because a greater amount of information for a scene is preserved by the HDR image compared to the separate SDR images.

[0004] Due to advances in design and manufacturing technology, display devices capable of displaying HDR images with a wider range of luminance values ​​are becoming more accessible. However, most display devices currently in use (and continue to be manufactured) are only capable of displaying SDR images with a more limited range of luminance values. Therefore, HDR images must be converted (i.e., downgraded) to be equivalent to SDR images displayed on devices that only have SDR-capable displays. Conversely, devices with HDR-capable displays may attempt to convert (i.e., upscale) SDR images to be equivalent to HDR images displayed via HDR-capable displays.

[0005] Existing conversion techniques may produce inconsistent and / or undesirable results. Specifically, downgrading an HDR image to an SDR image (which may be performed by a tone mapping operation) may introduce visual artifacts (e.g., banding) into the resulting SDR image that are typically uncorrectable using additional image processing. Conversely, upgrading an SDR image to an HDR image (which may be performed by an inverse tone mapping operation) involves applying different levels of guesswork, which may also introduce uncorrectable visual artifacts.

[0006] Furthermore, retaining all of the originally captured SDR images along with the HDR images may use up limited storage space and may require additional communication bandwidth to transfer all of the images between devices.

[0007] Therefore, what is needed is a technique for enabling images to be efficiently and accurately transformed between different states. For example, it is desirable to enable SDR images to be upgraded to HDR counterparts (and vice versa) without relying on the aforementioned (and flawed) conversion techniques. Summary of the invention

[0008] The embodiments described herein set forth techniques for preprocessing images to improve gain map compression results. The gain map can be generated by comparing a first image with a second image, and then compressed to form a compressed gain map. The compressed gain map can be combined with a compressed version of the first image to form a compressed enhanced image. The compressed enhanced image can be decompressed later and the gain map is applied to the first image to generate a version of the second image to provide reproduction on a target display. In some embodiments, the first image includes a standard dynamic range (SDR) image, and the second image includes a high dynamic range (HDR) image. In some embodiments, the first image includes an SDR image selected from a plurality of SDR images, and the second image includes an HDR image derived from a combination of the plurality of SDR images. In some embodiments, the gain map is determined by comparing the brightness values ​​of pixels in the HDR image with the brightness values ​​of corresponding pixels in the SDR image. In some embodiments, the full resolution version of the gain map includes the gain value of each pixel in the SDR image and the HDR image, and the reduced resolution version of the gain map includes the gain value of the group of two or more pixels in the SDR image and the HDR image. In some embodiments, the gain map is determined at 1 / 2 or 1 / 4 resolution compared to the SDR image and the HDR image. In some embodiments, the compressed gain map is generated by processing the gain map (at full resolution or at reduced resolution) using a first (gain map) compression scheme, and the compressed version of the first image is generated by processing the first image using a second (image) compression scheme. In some embodiments, the gain map is generated by comparing the brightness values ​​in the SDR image with the brightness values ​​in the HDR image. In some embodiments, the gain map is generated by comparing the brightness values ​​in the compressed version of the SDR image (or the compressed version of the HDR image) with the brightness values ​​in the (uncompressed) HDR image (or uncompressed SDR image). In some embodiments, the compressed enhanced image comprises a compressed version of the image derived from the SDR image and the HDR image jointly determined using a compressed version of the gain map (or jointly determined using an uncompressed version of the gain map that is subsequently compressed), the compressed version of the gain map being included with the compressed version of the image to form the compressed enhanced image. In some embodiments, the compressed enhanced image is stored in a non-volatile storage medium that is locally accessible by the computing device and / or remotely accessible by the computing device and possibly other computing devices (e.g., via a cloud network-based service).In some embodiments, a second computing device obtains the compressed enhanced image, extracts the compressed version of the image from the compressed enhanced image, generates an uncompressed version of the image from the compressed version of the image, generates an uncompressed version of the gain map from the compressed version of the gain map, and applies the uncompressed version of the gain map to the uncompressed version of the image to generate a second image formatted for display by the second computing device. The dynamic range of the brightness values ​​of the second image is different from the dynamic range of the brightness values ​​of the uncompressed version of the image. In some embodiments, the second image is an HDR image, and the uncompressed version of the image is an SDR image. In some embodiments, the compressed version of the first image and the compressed version of the gain map are generated by an image compression module and a gain map compression module, respectively, or jointly by a combined gain map generation and compression module. In some embodiments, the computing device generates a gain map, compresses the gain map to form a compressed version of the gain map, decompresses the compressed version of the gain map to generate an uncompressed version of the gain map, compares the (original) gain map with the uncompressed version of the gain map to determine an error map, and stores the error map with the compressed version of the gain map for use in creating the second image formatted for display by the second computing device.

[0009] Other embodiments include a non-transitory computer-readable storage medium configured to store instructions that, when executed by a processor included in a computing device, cause the computing device to perform the steps of any of the above methods. Additional embodiments include a computing device configured to perform various steps of any of the above methods.

[0010] Other aspects and advantages of the present invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings which illustrate by way of example the principles of the described embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The present disclosure will be more readily understood through the following detailed description taken in conjunction with the accompanying drawings, in which like reference numerals designate like structural elements.

[0012] Figure 1 An overview of a computing device that may be configured to perform the various techniques described herein is illustrated, according to some embodiments.

[0013] FIG. 2A to FIG. 2E A series of conceptual diagrams illustrating techniques for generating gain maps based on SDR images and HDR images according to some embodiments.

[0014] FIG. 3A to FIG. 3EA series of conceptual diagrams are illustrated for generating a compressed enhanced image based on a first image and a second image according to some embodiments.

[0015] Figure 4 A diagram illustrating an example of generating an HDR image for display by a computing device from a compressed enhanced image according to some embodiments.

[0016] Figure 5A and Figure 5B A flowchart of an exemplary method for image management by a computing device according to some embodiments is illustrated.

[0017] Figure 6 Illustrated is a detailed view of a computing device that can be used to implement the various techniques described herein, according to some embodiments. DETAILED DESCRIPTION

[0018] Representative applications of the methods and devices according to the present application are described in this section. These examples are provided only to add context and aid in understanding the described embodiments. Therefore, it will be apparent to those skilled in the art that the described embodiments may be practiced without some or all of these specific details. In other cases, in order to avoid unnecessarily obscuring the described embodiments, well-known processing steps are not described in detail. Other applications are possible, so that the following examples should not be considered limiting.

[0019] In the following detailed description, reference is made to the accompanying drawings which form a part of the specification and in which are shown by way of illustration specific embodiments in accordance with the described embodiments. Although these embodiments are described in sufficient detail to enable those skilled in the art to practice the described embodiments, it is to be understood that these examples are not limiting and other embodiments may be used and changes may be made without departing from the spirit and scope of the described embodiments.

[0020] Representative embodiments described herein disclose techniques for generating a gain map based on an acquired image. Specifically, the gain map can be generated by comparing a first image to a second image. The gain map can then be embedded in the first image so that the second image can be efficiently reproduced using the first image and the gain map. Figure 1 , FIG. 2A to FIG. 2E , FIG. 3A to FIG. 3E , Figure 4 , Figure 5A , Figure 5B and Figure 6 A more detailed description of these techniques is provided.

[0021] Figure 1An overview 100 of a computing device 102 that can be configured to perform the various techniques described herein is illustrated. Figure 1 As shown, computing device 102 may include processor 104, volatile memory 106, and non-volatile memory 124. It should be noted that FIG. 5 illustrates a more detailed breakdown of example hardware components that may be included in computing device 102, and is only for simplicity. Figure 1 For example, computing device 102 may include additional non-volatile memory (e.g., solid-state drive, hard disk drive, etc.), other processors (e.g., multi-core central processing unit (CPU)), graphics processing unit (GPU), etc. According to some embodiments, an operating system (OS) ( Figure 1 The OS may be loaded into the volatile memory 106, where the OS may execute a variety of applications that together enable the various techniques described herein. For example, these applications may include the image analyzer 110 (and its internal components), the gain map generator 120 (and its internal components), one or more compressors ( Figure 1 (not shown in the example) etc.

[0022] like Figure 1 As shown, the volatile memory 106 may be configured to receive a multi-channel image 108. The multi-channel image 108 may be, for example, acquired by a digital imaging unit ( Figure 1 According to some embodiments, the multi-channel image 108 may be composed of a set of pixels, wherein each pixel in the set of pixels includes a set of sub-pixels (e.g., red sub-pixels, green sub-pixels, blue sub-pixels, etc.). It should be noted that the term "sub-pixel" used herein may be synonymous with the term "channel". It should also be noted that the multi-channel image 108 may have different resolutions, layouts, bit depths, etc. without departing from the scope of the present disclosure.

[0023] According to some embodiments, a given multi-channel image 108 may represent a standard dynamic range (SDR) image, which constitutes a single exposure of a scene collected and processed by a digital imaging unit. A given multi-channel image 108 may also represent a high dynamic range (HDR) image, which constitutes multiple exposures of a scene collected and processed by a digital imaging unit. In order to generate an HDR image, the digital imaging unit may capture a scene at different exposure brackets (e.g., three exposure brackets commonly referred to as "EV0", "EV-", and "EV+"). Typically, an EVO image corresponds to a normal / ideal exposure of a scene (typically captured using an automatic exposure setting of a digital imaging unit); an EV- image corresponds to an underexposed image of a scene (e.g., four times darker than EV0), and an EV+ image corresponds to an overexposed image of a scene (e.g., four times brighter than EV0). The digital imaging unit may combine different exposures to produce a resulting image that combines a wider range of brightness relative to an SDR image. It should be noted that the multi-channel images 108 discussed herein are not limited to SDR / HDR images. Rather, multi-channel image 108 may represent any form of digital image (eg, a scanned image, a computer-generated image, etc.) without departing from the scope of the present disclosure.

[0024] like Figure 1 As shown, the multi-channel image 108 may (optionally) be provided to an image analyzer 110. According to some embodiments, the image analyzer 110 may include various components configured to process / modify the multi-channel image 108 as needed. For example, the image analyzer 110 may include a tone mapping unit 112 (e.g., configured to perform global / local tone mapping operations, inverse tone mapping operations, etc.), a noise reduction unit 114 (e.g., configured to reduce global / local noise in the multi-channel image), a color correction unit 116 (e.g., configured to perform global / local color correction in the multi-channel image), and a sharpening unit 118 (e.g., configured to perform global / local sharpening correction in the multi-channel image). It should be noted that the image analyzer 110 is not limited to the aforementioned processing units without departing from the scope of the present disclosure, and the image analyzer 110 may be combined with any number of processing units configured to perform any processing / modification on the multi-channel image 108.

[0025] like Figure 1As shown, after being processed by the image analyzer 110, the multi-channel image 108 may be provided to the gain map generator 120. However, it should be noted that, without departing from the scope of the present disclosure, the multi-channel image 108 may bypass the image analyzer 110 and be provided to the gain map generator 120 if desired. It should also be noted that, without departing from the scope of the present disclosure, the multi-channel image 108 may bypass one or more of the processing units of the image analyzer 110. For example, two given multi-channel images may pass through the tone mapping unit 112 to receive local tone mapping modifications and then bypass the remaining processing units in the image analyzer 110. In this regard, the two multi-channel images (which have undergone local tone mapping operations) may be used to generate a gain map 123 that reflects the local tone mapping operations performed. In any case, and as described in more detail herein, the gain map generator 120 may generate the gain map 123 based on the two multi-channel images 108 when the two multi-channel images 108 are received. Then, the gain map generator 120 may store the gain map 123 into one of the two multi-channel images 108 to produce the enhanced multi-channel image 122. It should also be noted that the gain map generation technique may be performed at any time relative to receiving the multi-channel image on which the gain map will be based. For example, the gain map generator 120 may be configured to postpone the generation of the gain map when the digital imaging unit is in active use in order to ensure that sufficient processing resources are available so that no slowdown is imposed on the user. FIG. 2A to FIG. 2E , Figure 3A To Figure 3G and Figure 4 A to Figure 4 E provides a more detailed breakdown of the manner in which the gain map generator 120 may generate the gain map 123 .

[0026] In addition, and although not in Figure 1 , but one or more compressors may be implemented on the computing device 102 for compressing the enhanced multi-channel image 122. For example, the compressor may implement a Lempel–Ziv–Welch (LZW) based compressor, other types of compressors, combinations of compressors, and the like. In addition, the compressor may be implemented in any manner to establish the most effective environment for compressing the enhanced multi-channel image 122. For example, multiple buffers (in which pixels may be preprocessed in parallel) may be instantiated, and each buffer may be connected to a corresponding compressor so that the buffers may also be compressed simultaneously in parallel. In addition, the same or different types of compressors may be connected to each of the buffers based on the formatting of the enhanced multi-channel image 122.

[0027] therefore, Figure 1Provides a high-level overview of different hardware / software architectures that can be implemented by the computing device 102 to implement the various techniques described herein. FIG. 2A to FIG. 2E Provides a more detailed breakdown of these techniques.

[0028] FIG. 2A to FIG. 2E A series of conceptual diagrams illustrating techniques for generating gain maps based on SDR images and HDR images according to some embodiments. Figure 2A As shown, step 210 may involve computing device 102 accessing a multi-channel HDR image 211, which is composed of pixels 212 (each denoted as "P"). Figure 2A As shown, the pixels 212 may be arranged according to a row / column layout, where a subscript "P" (eg, "1,1") of each pixel 212 indicates the position of the pixel 212 according to a row and a column. Figure 2A In the illustrated example, the pixels 212 of the multi-channel image 108 are arranged in an equal number of rows and columns, such that the multi-channel image 108 is a square image. However, it should be noted that the techniques described herein can be applied to multi-channel images 108 having different layouts (e.g., non-proportional row / column counts). In any case, and as Figure 2A As further shown, each pixel 212 may be composed of three sub-pixels 214: a red sub-pixel 214 (denoted as "R"), a green sub-pixel 214 (denoted as "G"), and a blue sub-pixel 214 (denoted as "B"). However, it should be noted that each pixel 212 may be composed of any number of sub-pixels without departing from the scope of the present disclosure.

[0029] Figure 2B Step 220 involving computing device 102 accessing a multi-channel SDR image 221 is illustrated. Figure 2B As shown, the multi-channel SDR image 221 is composed of Figure 2AThe illustrated multi-channel HDR image 211 is composed of pixels 222 (and sub-pixels 224) that are similar to the pixels 212 (and sub-pixels 214). According to some embodiments, the multi-channel SDR image 221 is captured by a single exposure of the same scene captured by the multi-channel HDR image 211, so that the multi-channel SDR image 221 and the multi-channel HDR image 211 are substantially related to each other. For example, if the multi-channel HDR image 211 is generated using the EV-, EV0, and EV+ methods described herein, the multi-channel SDR image 221 may be based on the EV0 exposure (e.g., before the EV0 exposure is merged with the EV-exposure and the EV+ exposure to generate the multi-channel HDR image 211). This method ensures that both the multi-channel HDR image 211 and the multi-channel SDR image 221 correspond to the same scene at the same time. In this way, the pixels of the multi-channel HDR image 211 and the multi-channel SDR image 221 may differ only in the luminosity collected from the same point of the same scene (as opposed to the scene content being different due to motion resulting from the passage of time that would occur through sequentially captured exposures).

[0030] Figure 2C Step 230 is illustrated, which involves computing device 102 comparing multi-channel HDR image 211 and multi-channel SDR image 221 (in Figure 2C 234) to generate a multi-channel gain map 231 (composed of pixels 232). Here, if it is desired to enable the use of the multi-channel HDR image 211 to reproduce the multi-channel SDR image 221, a first method may be used. Specifically, the first method involves dividing the value of each pixel of the multi-channel SDR image 221 by the value of the corresponding pixel of the multi-channel HDR image 211 to produce a quotient. Then, the corresponding quotient may be assigned to the value of the corresponding pixel 232 in the multi-channel gain map 231. For example, if the representation of the multi-channel HDR image 211 is “P 1,1 ” has a value of “5”, and the representation of the multi-channel SDR image 221 is “P 1,1 ” has a value of “1”, the quotient will be “0.2”, and the representation assigned to the multi-channel gain map 231 will be “P 1,1 In this manner, and as described in more detail herein, the representation of the multi-channel SDR image 221 is “P 1,1 The pixels of the multi-channel HDR image 211 can be represented by “P 1,1 ” (having a value of “5”) is multiplied by the representation of the multi-channel gain map 231 as “P 1,1 ” is reproduced. Specifically, the multiplication will generate a product “1”, which is represented as “P 1,1Therefore, storing the multi-channel gain map 231 together with the multi-channel HDR image 221 may enable the multi-channel SDR image 221 to be reproduced independently of the multi-channel SDR image 211 itself. Figure 2D A more detailed description of the various ways in which the multi-channel gain map 231 may be stored with the corresponding multi-channel image is described.

[0031] Alternatively, if it is desired to enable the use of multi-channel SDR image 221 to reproduce multi-channel HDR image 211, a second (different) method may be utilized. Specifically, the second method involves dividing the value of each pixel of multi-channel HDR image 211 by the value of the corresponding pixel of multi-channel SDR image 221 to produce a quotient. The corresponding quotient may then be assigned to the value of the corresponding pixel 232 in multi-channel gain map 231. For example, if the representation of multi-channel SDR image 221 is “P 1,1 ” has a value of “3”, and the representation of the multi-channel HDR image 211 is “P 1,1 ” has a value of “6”, the quotient will be “2”, and the representation assigned to the multi-channel gain map 231 will be “P 1,1 In this manner, and as described in more detail herein, the representation of the multi-channel HDR image 211 is “P 1,1 The pixels of the multi-channel SDR image 221 can be represented by “P 1,1 ” (having a value of “3”) is multiplied by the representation of the multi-channel gain map 231 as “P 1,1 ” is reproduced. Specifically, the multiplication will generate a product “6”, which is represented by “P 1,1 ” matches the value “6” of the pixel of ”. Therefore, storing the multi-channel gain map 231 together with the multi-channel SDR image 221 may enable the multi-channel HDR image 211 to be reproduced independently of the multi-channel HDR image 211 itself. Figure 2D A more detailed description of the various ways in which the multi-channel gain map 231 may be stored with the corresponding multi-channel image is described.

[0032] In short, it should be noted that although Figure 2C The comparisons illustrated in (and described herein) constitute pixel-level comparisons, but the embodiments are not limited thereto. Rather, pixels of the images may be compared to one another at any level of granularity without departing from the scope of the present disclosure. For example, sub-pixels of the multi-channel HDR image 211 and the multi-channel SDR image 221 may be compared to one another (instead of or in addition to pixel-level comparisons) such that multiple gain maps (e.g., corresponding gain maps for each color channel) are generated under different comparison methods.

[0033] In addition, it should be noted that various optimizations may be employed when generating a gain map without departing from the scope of the present disclosure. For example, when two values ​​are identical to each other, the comparison operation may be skipped, and a single bit value (e.g., "0") may be assigned to the corresponding value in the gain map to minimize the size of the gain map (i.e., storage requirements). In addition, the resolution of the gain map may be less than the resolution of the image that is compared to generate the gain map. For example, an approximation of every four pixels in the first image may be compared with an approximation of every four corresponding pixels in the second image to generate a gain map that is one-fourth the resolution of the first image and the second image. This method will significantly reduce the size of the gain map, but will reduce the overall accuracy with which the first image can be reproduced based on the second image and the gain map (or vice versa). In addition, the first image and the second image may be resampled in any conceivable manner before generating the gain map. For example, the first image and the second image may undergo a local tone mapping operation before generating the gain map. In some embodiments, the first image and the second image used to generate the gain map may undergo a local tone mapping operation before generating the gain map. In some embodiments, a global tone map is generated and used with a gain map that provides a local adaptive tone map. In some embodiments, the gain map is stored at multiple resolutions (or as a multi-scale image), and different gain map values ​​may be obtained from the gain map for use with images of different sizes to be displayed using the gain map.

[0034] Figure 2D Step 240 according to some embodiments is illustrated, which involves the computing device 102 embedding the multi-channel gain map 231 into the multi-channel HDR image 211 or the multi-channel SDR image 221. Specifically, if the above combined Figure 2C In the first method discussed (which enables the use of the multi-channel HDR image 211 and the multi-channel gain map 231 to reproduce the multi-channel SDR image 221), the computing device 102 embeds the multi-channel gain map 231 into the multi-channel HDR image 211 (thereby producing the enhanced multi-channel image 122). Figure 2DAs shown, one method for embedding the multi-channel gain map 231 into the multi-channel HDR image 211 involves interleaving each pixel 232 (of the multi-channel gain map 231) with the corresponding pixel 212 (of the multi-channel HDR image 211). An alternative method may involve embedding each pixel 232 (of the multi-channel gain map 231) into the corresponding pixel 212 (of the multi-channel gain map 231) as an additional channel of the pixel 212. Yet another method may involve embedding the multi-channel gain map 231 as metadata stored with the multi-channel HDR image 211. It should be noted that the aforementioned methods are exemplary and not intended to be limiting, and any conceivable method may be used to store the multi-channel gain map 231 (and other supplemental gain maps, if generated) with the multi-channel HDR image 211 without departing from the scope of the present disclosure.

[0035] Figure 2E A method 250 for generating a gain map based on an SDR image and an HDR image according to some embodiments is illustrated. Figure 2E As shown in FIG. 2 , method 250 begins at step 252, where computing device 102 accesses an HDR image (e.g., as described above in conjunction with Figure 2A At step 254, computing device 102 accesses the SDR image (e.g., as described above in conjunction with Figure 2B At step 256, computing device 102 generates a gain map by comparing the HDR image to the SDR image, or vice versa (e.g., as described above in conjunction with Figure 2C In step 258, computing device 102 embeds the gain map into the HDR image or SDR image (e.g., as described above in conjunction with Figure 2D As described, an enhanced multi-channel image 122 is produced).

[0036] Figure 3A Illustrated Figure 1300 of a computing device 102 having an additional computing module to generate a compressed enhanced multi-channel image 308 from an enhanced multi-channel image 122, the generation of which is previously described herein. The enhanced multi-channel image 122 includes a gain map 123 generated by comparing at least two multi-channel images 108 paired with one of the multi-channel images 108. In some embodiments, the compression module 302 processes the multi-channel image 108 (which may be in an uncompressed form) alone or in conjunction with the gain map 123 to form a compressed multi-channel image 304. The compression module 302 also processes the gain map 123 (which may be in an uncompressed form) alone or in conjunction with the multi-channel image 108 to form a compressed gain map 306. The compressed multi-channel image 304 may be combined with the compressed gain map 306 to form a compressed enhanced multi-channel image 308, which may be stored locally, for example, in the non-volatile memory 124 (or another local storage medium) of the computing device 102 and / or may be stored remotely, for example, in a cloud-based service (such as provided by Management In some embodiments, the remotely stored compressed enhanced multi-channel image 308 may be obtained by the second computing device 102 and used to generate an uncompressed image having a dynamic range of brightness values ​​suitable for display by the second computing device 102.

[0037] Figure 3BA diagram 310 for generating a compressed enhanced multi-channel image 324 by a computing device 102 is illustrated. The gain map generator 120 may compare luminance values ​​of pixels in a first multi-channel image 108-A with luminance values ​​of pixels in a second multi-channel image 108-B to generate a gain map 123. The gain map 123 may be defined based on a bit depth (e.g., at least 10 bits) of each gain map value and a gain map resolution, which may include downsampling to reduce storage requirements for the gain map 123. In some embodiments, the gain map 123 is reduced from a full resolution, in which each gain map value in the gain map 123 corresponds to a single pixel in each of the first multi-channel image 108-A and the second multi-channel image 108-B, to a lower resolution, in which each gain value in the gain map 123 corresponds to multiple pixels in each of the first multi-channel image 108-A and the second multi-channel image 108-B. When the gain map 123 is used together with the uncompressed version of the compressed multi-channel image 320 having the first dynamic range to generate a second image having a second dynamic range for display, reducing the resolution of the gain map 123 may affect the image quality. The reduction in the resolution of the gain map 123 generally results in a reduction in the contrast of the image subsequently generated using the gain map 123. In some cases, the 1 / 2 or 1 / 4 resolution of the gain map 123 (the latter represents the 1 / 2 resolution in each dimension of the image) has a limited impact on the image quality of the image subsequently generated using the gain map 123. The gain map 123 provides a representation of the brightness difference between images having different dynamic ranges (e.g., between an SDR image and an HDR image). In some cases, the first multi-channel image 108-A is an ideal (or reference) SDR image of the scene, and the second multi-channel image 108-B is an ideal (or reference) HDR image of the scene. The ideal SDR image and HDR image may be manually generated by a user of the computing device 102, for example, using an image processing application, or may be automatically generated by an image capture application of the computing device 102. The gain map 123 is intended to allow only one image of a scene, such as an SDR image (or a version derived therefrom) or an HDR image (or a version derived therefrom), to be stored, and a corresponding complementary image of the scene to be subsequently generated. For example, an SDR image may be stored with the gain map 123, and later an HDR image may be generated by applying the gain map 123 to the SDR image. Depending on its resolution, the gain map 123 allows a local tone map to be applied to a smaller area of ​​the image than a global tone map would be applied to the entire image. A global tone map affects the entire image, where each pixel of the image is mapped using the same function for each pixel, without regard to the local context of nearby pixels.Local tone mapping affects a local area of ​​an image, and considers pixels adjacent to individual pixels to determine the mapping function, and can improve the contrast between adjacent pixels compared to global tone mapping. In the embodiments described herein, the main goal is to generate a base image derived from multiple (usually two) images and a gain map 123 that captures the difference between the multiple images, each of which is optimized for different dynamic ranges of brightness. In some embodiments, the base image can be used to generate a first display image with a first dynamic range, such as an SDR display image, and the base image together with the gain map can be used to generate a second display image with a second dynamic range, such as an HDR display image. A full-resolution gain map 123 including a gain value for each pixel in the associated base image can provide a high quality level, but requires a large amount of storage. In some embodiments, the base image is an SDR image, and multiple gain maps 123 are generated, each of which is associated with a different HDR display capability. Storing multiple gain maps 123 at full resolution together with the SDR image may require more storage than the user of the computing device 102 expects (or can obtain). Reducing the resolution of the gain map 123 provides one form of storage reduction; however, overly aggressive resolution reduction of the gain map 123 may lead to unnatural results when the HDR image is later regenerated for display from the SDR (base) image and the gain map 123. Compression of the base image and the gain map 123 (with possibly a modest resolution reduction of the gain map 123, such as 1 / 2 resolution corresponding to gain map values ​​for every pair of pixels or 1 / 4 resolution corresponding to gain map values ​​for every quarter pixel) may provide compact storage and high quality results.

[0038] In a first specific implementation of image and gain map compression, as Figure 3BAs illustrated, the gain map 123 generated by the gain map generator 120 from the first multi-channel image 108-A and the second multi-channel image 108-B is processed by the gain map compression module 312 to generate a compressed gain map 322. The first multi-channel image 108-A or the second multi-channel image 108-B is selected by the image selection module 314, respectively, and the selected multi-channel image 316 is processed by the image compression module 318 to produce a compressed multi-channel image 320. The compressed multi-channel image 320 can be combined with the compressed gain map 322 to form a compressed enhanced multi-channel image 324, which can be stored locally at the computing device 102 or remotely at an external storage device, such as at a cloud network-based service accessible to the computing device 102. The compressed multi-channel image 320 can later be decompressed (or uncompressed) to copy the selected multi-channel image 316 (which can be the first multi-channel image 108-A or the second multi-channel image 108-B). The compressed gain map 322 may also be decompressed (or uncompressed) to reproduce the gain map 123, which may be combined with the uncompressed version of the compressed multi-channel image 320 to produce a version of the first or second (i.e., unselected) multi-channel image 108-A, 108-B. In some embodiments, the image compression module 318 uses an image compression algorithm optimized for processing images, while the gain map compression module 312 uses a gain map compression algorithm optimized for processing the gain map 123, which may have characteristics that differ significantly from those of an image.

[0039] Figure 3C Graph 330 illustrates another technique for generating a compressed enhanced multi-channel image 338. Gain map generator 120 processes first multi-channel image 108-A and second multi-channel image 108-B to generate gain map 123, which is processed by gain map compression module 312 to form compressed gain map 306. First multi-channel image 108-A and second multi-channel image 108-B are jointly processed by image compression module 332 to form compressed multi-channel image 334. Compressed multi-channel 334 and compressed gain map 306 are combined to form compressed multi-channel image 338. Figure 3B In the technique, the first multi-channel image 108-A or the second multi-channel image 108-B is selected and compressed to form a compressed multi-channel image 320, and Figure 3CIn the technique, the first multi-channel image 108-A and the second multi-channel image 108-B are processed together to generate a compressed multi-channel image 334. In some embodiments, the compressed multi-channel image 334 may be decompressed (or uncompressed) to form a version of the first multi-channel image 108-A or the second multi-channel image 108-B. In some embodiments, the compressed multi-channel image 334 may be decompressed (or uncompressed) and combined with a decompressed version of the gain map 123 obtained from the compressed gain map 306 to form the second multi-channel image 108-B or a corresponding complementary version of the first multi-channel image 108-A. The first multi-channel image 108-A and the second multi-channel image 108-B may have brightness values ​​with different dynamic ranges, and the corresponding reconstructed versions of the first multi-channel image 108-A and the second multi-channel image 108-B may also have brightness values ​​with different dynamic ranges.

[0040] Figure 3D Diagram 340 illustrates another technique for generating a compressed enhanced multi-channel image 344. A first multi-channel image 108-A (which may be an SDR image or an HDR image) may be processed by an image compression module 318 to generate a compressed first multi-channel image 342. A second multi-channel image 108-B (which may be a complementary HDR image or a complementary SDR image) may be processed by a gain map generator 120 along with the compressed first multi-channel image 342 to generate a gain map 123. The gain map 123 may then be processed by a gain map compression module 312 to form a compressed gain map 322, which may be combined with the compressed first multi-channel image 342 to form a compressed enhanced multi-channel image 344, which may be stored locally at the computing device 102 or remotely at an accessible storage facility separate from the computing device 102, such as at a server based on a cloud network. The compressed enhanced multi-channel image 344 may be obtained by the computing device 102 (or in some cases by another computing device 102) from local or remote storage and used to regenerate versions of the first multi-channel image 108-A and the second multi-channel image 108-B. The compressed first multi-channel image 342 may be extracted from the compressed enhanced multi-channel image 344 and decompressed (or uncompressed) to replicate a version of the first multi-channel image 108-A. The compressed gain map 322 may be extracted from the compressed enhanced multi-channel image 344 and decompressed (or uncompressed) to obtain a version of the gain map 123, which may be combined with a version of the first multi-channel image 108-A to generate a version of the second multi-channel image 108-B suitable for display.

[0041] In some cases, FIG. 3B to FIG. 3DThe embodiment shown in , in which the selected multi-channel image 316 and the gain map 123 are compressed independently, is not ideal because the selected multi-channel image 316 and the gain map 123 may be strongly correlated with each other. Independent compression and subsequent decompression (or uncompression) of the compressed multi-channel image 320 and the compressed gain map 322, followed by application of the decompressed gain map to the decompressed multi-channel image, may result in compression artifacts in the resulting image. For example, compression artifacts affecting gain map pixels (or sets of pixels) and separate compression artifacts affecting image pixels (or sets of pixels) may result in significant errors when decompressing the gain map and applying the gain map 123 to the decompressed image. Improved implementations of compression may include joint (or closed-loop) compression that uses a combination of the image and the gain map 123 to generate a compressed version included in a compressed enhanced multi-channel image.

[0042] Figure 3E A diagram 350 of another example of generating a compressed enhanced multi-channel image 358 is illustrated. A combined (joint) gain map generation and compression module 352 jointly processes the first multi-channel image 108-A and the second multi-channel image 108-B to form a compressed enhanced multi-channel image 358, which includes a compressed multi-channel image 354 and a compressed gain map 356. The first multi-channel image 108-A and the second multi-channel image 108-B may each have a different dynamic range of brightness values, for example, the first multi-channel image 108-A may be an SDR multi-channel image, and the second multi-channel image 108-B may be an HDR multi-channel image. In some embodiments, a version of one of the first multi-channel image 108-A and the second multi-channel image 108-B may be generated using the compressed multi-channel image 354, while a version of the other of the first multi-channel image 108-A and the second multi-channel image 108-B may be generated using the compressed multi-channel image 354 in combination with the compressed gain map 356.

[0043] In some embodiments, additional metadata is generated and stored with the compressed enhanced multi-channel images 308, 324, 338, 344, 358. Exemplary metadata includes content information, such as whether the image includes a human face, the maximum amount of headroom available for processing the image content, offset values ​​for the gain map 123, and / or an error map associated with compression artifacts of the compressed gain map.

[0044] Figure 4Diagram 400 illustrates an example of an HDR multi-channel image 426 generated from a compressed enhanced multi-channel image 406 for display by a computing device 102. The compressed enhanced multi-channel image 406 may have been previously generated by a computing device 102 or by a separate computing device 102 that decompresses and generates the HDR multi-channel image 426 targeted for display. For example, the compressed multi-channel image 406 may be generated on a first computing device 102, stored at a cloud network-based server, retrieved by a second computing device 102, and processed by the second computing device 102 for presentation on a display associated with the second computing device 102. The generated HDR multi-channel image 426 displayed by the second computing device 102 may be processed according to known properties of the display, which may not have been known when the compressed enhanced multi-channel image 406 was generated by the first computing device 102.

[0045] The computing device 102 may extract the compressed multi-channel image 402 from the compressed enhanced multi-channel image 406, decompress the extracted compressed enhanced multi-channel image 406 to generate a multi-channel image base layer 408, which may be in an SDR format in some embodiments. The computing device 102 may also extract the compressed gain map 404 from the compressed enhanced multi-channel image 406, decompress the extracted compressed gain map 404 to generate an uncompressed version of the gain map 410. The gain map 410 may be processed by a renormalization module 414, which considers minimum and maximum logarithmic (log2) values ​​418 when processing for different color channels. In some cases, the gain map values ​​of the red channel are scaled and processed in the logarithmic domain, including via an exponential function module 416, while the gain map values ​​of the cyan channel are scaled and processed in the linear domain. The values ​​of the gain map 410 may be appropriately scaled in a gain map scaling module 428 using knowledge of the peak values ​​430 of the display on which the final HDR multi-channel image 426 is intended to be displayed. The scaled gain map values ​​for the color channels may be applied to the multi-channel image base layer 408 (after passing through an applicable de-gamma function module 412) at a gain mapping module 422, which also uses the offset values ​​420 previously stored as metadata with the compressed gain map 404. The output of the gain mapping module 422 is further processed by a color management module 424 to produce an HDR multi-channel image optimized for a particular display. In some embodiments, metadata (such as the offset values ​​420 and the minimum and maximum log2 values ​​418) are stored with the gain map 410 (and compressed with the gain map 410) or stored with the compressed gain map 404. In some embodiments, the gain map 410 uses normalized values ​​having a valid value range from 0 to 1, and the renormalized version of the gain map 410 includes the full range of gain map values ​​that were initially calculated when determining the original version of the gain map 410 (when comparing the original SDR image and the HDR image). In some embodiments, the renormalized gain map values ​​are log2 scaled values, and the linear version of the renormalized gain map values ​​are exponential values, e.g., a log2 scaled value x corresponds to a linear scaled value 2 xIn some embodiments, the scaling of a portion of the gain map 410 by the gain map scaling module 428 occurs in the logarithmic domain (such as for certain color channels). In some embodiments, the scaling of a portion of the gain map 410 by the gain map scaling module 428 occurs in the linear domain (such as for certain other color channels). In some embodiments, the amount of scaling used to generate the scaled gain map values ​​to be applied to the base layer image (after the degamma module 412) is based on the capabilities of the target display, display environment conditions (e.g., brighter or darker ambient light), and / or other metadata values. In some embodiments, the gain map 410 is generated at the source computing device 102 by calculating the ratio of the pixel brightness values ​​and adding the offset value 420 to the divisor pixel brightness value of zero to ensure that division by zero does not occur in the ratio calculation. Then, when the regenerated gain map is applied (at the gain map module 422), the offset value 420 can be removed. In some embodiments, the offset value 420 can be selected based on the original SDR image, the original HDR image, and / or based on compression and / or gain mapping considerations. In some embodiments, the offset value 420 is selected to optimize gain map storage.

[0046] In some embodiments, the computing device 102 determines a gain map 123, compresses the gain map 123 to form a compressed gain map 306, 322, 356, decompresses the compressed gain map 306, 322, 356 to form an uncompressed version of the gain map 123, and compares values ​​in the (original) gain map 123 with values ​​in the uncompressed version of the gain map 123 to determine an error map that captures errors with the compression of the gain map 123. The computing device 102 may store the error map with the compressed enhanced multi-channel image 308, 324, 338, 344, 358 (either with the compressed gain map 306, 322, 356 or separately with accompanying metadata). In some embodiments, computing device 102 determines multiple gain maps 123, each gain map 123 is intended for a different purpose, such as for a different target display with different characteristics (e.g., size, resolution, color gamut range, maximum brightness), or to a later rendered image derived from a base image and a gain map, each image having different stylistic characteristics, i.e., a different version of an image. For example, computing device 102 may generate multiple gain maps 123 for different peak display values ​​of 500 nits, 1000 nits, 2000 nits, and 4000 nits, and multiple gain maps 123 may be used to generate different images optimized for different displays with different peak display values. In some embodiments, computing device 102 generates gain map 123 for multi-channel image 108, and then transcodes multi-channel image 108 into another image format that is different from the image format used for the original multi-channel image 108, such as when changing the color space used for the image. The computing device 102 may recalculate the gain map for the transcoded multi-channel image 108 from the original gain map 123 or a gain map newly calculated based on the transcoded multi-channel image 108 .

[0047] In some embodiments, the computing device 102 determines a gain map 123 at multiple resolutions, such as a multi-scale gain map, which can be compressed and stored with a base image, which can also be at multiple resolutions or can be resampled to multiple resolutions, and an appropriate gain map can be derived from the multi-scale gain map to be applied to the base image (or a resampled version of the base image) to provide an image for display at the relevant resolution. Thus, the multi-scale gain map can be used to apply local adaptive tone mapping to images sized for different output displays. In some embodiments, a global tone map is applied to a baseline image before (or after) applying the gain map to obtain an image for display, wherein the gain map provides an adaptive local tone map separate from the global tone map.

[0048] Figure 5AFlowchart 500 of an exemplary method for image management by computing device 102 is illustrated. At 502, a computing device generates a compressed version of an image and a compressed version of a gain map from a standard dynamic range (SDR) image of a scene and a high dynamic range (HDR) image of the scene. At 504, computing device 102 combines the compressed version of the image with the compressed version of the gain map to form a compressed enhanced image. At 506, computing device 102 stores the compressed enhanced image in a non-volatile storage medium.

[0049] Figure 5B A flowchart 520 of another exemplary method of image management by a second computing device 102 is illustrated. At 522, the second computing device 102 obtains a compressed enhanced image. At 524, the second computing device 102 extracts a compressed version of the image and a compressed version of the gain map from the compressed enhanced image. At 526, the second computing device generates an uncompressed version of the image from the compressed version of the image. At 528, the second computing device generates an uncompressed version of the gain map from the compressed version of the gain map. At 530, the second computing device applies the uncompressed version of the gain map to the uncompressed version of the image to generate a second image formatted for display by the second computing device 102, wherein the second image and the uncompressed version of the image have luminance values ​​of different dynamic ranges.

[0050] In some embodiments, the compressed image comprises a compressed version of the SDR image. In some embodiments, the second image comprises a version of the HDR image. In some embodiments, the compressed image comprises a compressed version of the HDR image. In some embodiments, the second image comprises a version of the SDR image. In some embodiments, the method performed by the computing device 102 also includes the computing device 102: i) generating a gain map by comparing the brightness values ​​of pixels in the HDR image with the brightness values ​​of corresponding pixels in the SDR image, ii) generating a compressed version of the image by processing the SDR image or the HDR image using an image compression module, and iii) generating a compressed version of the gain map by processing the gain map using a gain compression module. In some embodiments, the method performed by the computing device 102 also includes the computing device 102: i) generating a gain map by comparing the brightness values ​​of pixels in the HDR image with the brightness values ​​of corresponding pixels in the SDR image, ii) generating a compressed version of the image by jointly processing the SDR image and the HDR image using an image compression module, and iii) generating a compressed version of the gain map by processing the gain map using a gain compression module. In some embodiments, the method performed by the computing device 102 also includes the computing device 102: i) generating a compressed version of the image by processing the SDR image using an image compression module, ii) generating a gain map by comparing the brightness values ​​of pixels in the HDR image with the brightness values ​​of corresponding pixels in the compressed version of the image, and iii) generating a compressed version of the gain map by processing the gain map using a gain compression module. In some embodiments, the computing device 102 generates a compressed version of the image and a compressed version of the gain map by jointly generating a compressed version of the gain map and a compressed version of the image from the SDR image and the HDR image using a combined gain map generation and compression module. In some embodiments, the compressed version of the gain map is derived from a gain map having a linear resolution in each of two dimensions that is the same as the linear resolution of the SDR image and the HDR image. In some embodiments, the compressed version of the gain map is derived from a gain map having a linear resolution in at least one dimension that is less than the corresponding linear resolution of the SDR image and the HDR image. In some embodiments, the compressed version of the gain map is generated using a first compression scheme optimized for the gain map, and the compressed version of the image is generated using a second compression scheme optimized for the image.

[0051] Figure 6 A detailed view of a computing device 600 that can be used to implement the various components described herein is illustrated according to some embodiments. In particular, the detailed view illustrates a combination of Figure 1 The various components that may be included in the computing device 102 are described. Figure 6As shown, the computing device 600 may include a processor 602 representing a microprocessor or controller for controlling the overall operation of the computing device 600. The computing device 600 may also include a user input device 608 that allows a user of the computing device 600 to interact with the computing device 600. For example, the user input device 608 may take a variety of forms, such as buttons, keypads, dials, touch screens, audio input interfaces, visual / image capture input interfaces, input in the form of sensor data, and the like. In addition, the computing device 600 may include a display 610 that may be controlled by the processor 602 (e.g., via a graphics component) to display information to the user. A data bus 616 may facilitate data transmission between at least a storage device 640, the processor 602, and a controller 613. The controller 613 may be used to interface with and control different equipment through an equipment control bus 614. The computing device 600 may also include a network / bus interface 611 coupled to the data link 612. In the case of a wireless connection, the network / bus interface 611 may include a wireless transceiver.

[0052] As noted above, the computing device 600 also includes a storage device 640, which may include a single disk (e.g., a hard disk) or a collection of disks. In some embodiments, the storage device 640 may include flash memory, semiconductor (solid-state) memory, etc. The computing device 600 may also include a random access memory (RAM) 620 and a read-only memory (ROM) 622. The ROM 622 may store programs, utilities, or processes to be executed in a non-volatile manner. The RAM 620 may provide volatile data storage and store instructions related to the operation of an application program (e.g., image analyzer 110 / gain map generator 120) executed on the computing device 600.

[0053] The techniques described herein include techniques for image management. According to some embodiments, a first technique may be implemented by a computing device and includes the following steps: (1) generating a compressed version of an image and a compressed version of a gain map from a standard dynamic range (SDR) image of a scene and a high dynamic range (HDR) image of the scene; (2) combining the compressed version of the image with the compressed version of the gain map to form a compressed enhanced image; and (3) storing the compressed enhanced image in a non-volatile storage medium.

[0054] According to some embodiments, the above technology may also include the following steps performed by a second computing device: (1) obtaining a compressed enhanced image; (2) extracting a compressed version of the image and a compressed version of the gain map from the compressed enhanced image; (3) generating an uncompressed version of the image from the compressed version of the image; (4) generating an uncompressed version of the gain map from the compressed version of the gain map; and (5) applying the uncompressed version of the gain map to the uncompressed version of the image to generate a second image formatted for display by a second computing device, wherein the second image and the uncompressed version of the image have brightness values ​​with different dynamic ranges.

[0055] According to some embodiments, the compressed version of the image comprises a compressed version of the SDR image; and the second image comprises a version of the HDR image. According to some embodiments, the compressed version of the image comprises a compressed version of the HDR image; and the second image comprises a version of the SDR image. According to some embodiments, the compressed version of the gain map is generated using a lossy compression module selected for use with gain map compression. According to some embodiments, the compressed version of the image is generated using a lossy compression module selected for use with image compression.

[0056] According to some embodiments, generating a compressed version of an image and a compressed version of a gain map includes: (1) generating a gain map by comparing the brightness values ​​of pixels in an HDR image with the brightness values ​​of corresponding pixels in an SDR image; (2) generating a compressed version of the image by processing the SDR image or the HDR image using an image compression module; and (3) generating a compressed version of the gain map by processing the gain map using a gain compression module.

[0057] According to some embodiments, generating a compressed version of an image and a compressed version of a gain map includes: (1) generating a gain map by comparing the brightness values ​​of pixels in an HDR image with the brightness values ​​of corresponding pixels in an SDR image; (2) generating a compressed version of the image by jointly processing the SDR image and the HDR image using an image compression module; and (3) generating a compressed version of the gain map by processing the gain map using a gain compression module.

[0058] According to some embodiments, generating a compressed version of an image and a compressed version of a gain map includes: (1) generating a compressed version of the image by processing an SDR image using an image compression module; (2) generating a gain map by comparing the brightness values ​​of pixels in the HDR image with the brightness values ​​of corresponding pixels in the compressed version of the image; and (3) generating a compressed version of the gain map by processing the gain map using a gain compression module.

[0059] According to some embodiments, generating a compressed version of the image and a compressed version of the gain map includes: using a combined gain map generation and compression module to jointly generate a compressed version of the gain map and a compressed version of the image from the SDR image and the HDR image.

[0060] According to some embodiments, a compressed version of the gain map is derived from a gain map having a linear resolution in each of two dimensions that is the same as the linear resolution of the SDR image and the HDR image. According to some embodiments, a compressed version of the gain map is derived from a gain map having a linear resolution in at least one dimension that is less than the corresponding linear resolution of the SDR image and the HDR image. According to some embodiments, the compressed version of the gain map is generated using a first compression scheme optimized for the gain map; and the compressed version of the image is generated using a second compression scheme optimized for the image.

[0061] According to some embodiments, the computing device uses offset values ​​based on pixel values ​​of the SDR image and / or pixel values ​​of the HDR image when generating the gain map. According to some embodiments, the computing device selects the offset values ​​to optimize storage of a compressed version of the gain map.

[0062] According to some embodiments, the above technique may further include the following steps: generating, by a computing device, an uncompressed version of the gain map from the compressed version of the gain map; (2) determining an error map based on comparing the uncompressed version of the gain map with an original gain map used to generate the compressed version of the gain map; and (3) storing the compressed version of the error map with the compressed enhanced image. According to some embodiments, the compressed version of the error map is compressed using a lossless compression module; and the compressed version of the gain map is compressed using a lossy compression module.

[0063] The various aspects, embodiments, implementations or features of the described embodiments may be used individually or in any combination. The various aspects of the described embodiments may be implemented by software, hardware, or a combination of hardware and software. The described embodiments may also be embodied as computer readable code on a computer readable medium. A computer readable medium is any data storage device that can store data, which can then be read by a computer system. Examples of the computer readable medium include read-only memory, random access memory, CD-ROM, DVD, magnetic tape, hard disk drive, solid state drive, and optical data storage device. The computer readable medium may also be distributed in a network coupled computer system so that the computer readable code is stored and executed in a distributed manner.

[0064] For the purpose of explanation, the foregoing description uses specific nomenclature to provide a thorough understanding of the described embodiments. However, it will be apparent to those skilled in the art that specific details are not required in order to practice the described embodiments. Therefore, the foregoing description of specific embodiments is presented for the purpose of illustration and description. The foregoing description is not intended to be exhaustive or to limit the described embodiments to the precise form disclosed. It will be apparent to those of ordinary skill in the art that, in view of the above teachings, many modifications and variations are possible.

Claims

1. A method for image management, the method comprising, at a computing device: generating a compressed version of the image and a compressed version of the gain map from a standard dynamic range (SDR) image of the scene and a high dynamic range (HDR) image of the scene; combining the compressed version of the image with the compressed version of the gain map to form a compressed enhanced image; as well as The compressed enhanced image is stored in a non-volatile storage medium.

2. The method of claim 1 , further comprising, at the second computing device: obtaining the compressed enhanced image; extracting the compressed version of the image and the compressed version of the gain map from the compressed enhanced image; generating an uncompressed version of the image from the compressed version of the image; generating an uncompressed version of the gain map from the compressed version of the gain map; as well as applying the uncompressed version of the gain map to the uncompressed version of the image to generate a second image formatted for display by the second computing device, Wherein the second image and the uncompressed version of the image have brightness values ​​with different dynamic ranges.

3. The method according to claim 2, in: The compressed version of the image comprises a compressed version of the SDR image; and The second image comprises a version of the HDR image.

4. The method according to claim 2, in: The compressed version of the image comprises a compressed version of the HDR image; and The second image comprises a version of the SDR image.

5. The method of claim 1, wherein the compressed version of the gain map is generated using a lossy compression module selected for use with gain map compression.

6. The method of claim 1, wherein the compressed version of the image is generated using a lossy compression module selected for use with image compression.

7. The method of claim 1 , wherein the compressed version of the image and the compressed version of the gain map are generated include: generating a gain map by comparing brightness values ​​of pixels in the HDR image with brightness values ​​of corresponding pixels in the SDR image; generating the compressed version of the image by processing the SDR image or the HDR image using an image compression module; as well as The compressed version of the gain map is generated by processing the gain map using a gain compression module.

8. The method of claim 1 , wherein the compressed version of the image and the compressed version of the gain map are generated include: generating a gain map by comparing brightness values ​​of pixels in the HDR image with brightness values ​​of corresponding pixels in the SDR image; generating the compressed version of the image by jointly processing the SDR image and the HDR image using an image compression module; as well as The compressed version of the gain map is generated by processing the gain map using a gain compression module.

9. The method of claim 1 , wherein the compressed version of the image and the compressed version of the gain map are generated include: generating the compressed version of the image by processing the SDR image using an image compression module; generating a gain map by comparing luminance values ​​of pixels in the HDR image with luminance values ​​of corresponding pixels in the compressed version of the image; as well as The compressed version of the gain map is generated by processing the gain map using a gain compression module.

10. The method of claim 1, wherein the compressed version of the image and the compressed version of the gain map are generated include: The compressed version of the gain map and the compressed version of the image are jointly generated from the SDR image and the HDR image using a combined gain map generation and compression module.

11. The method of claim 1 , wherein the compressed version of the gain map is derived from a gain map having a linear resolution in each of two dimensions that is the same as a linear resolution of the SDR image and the HDR image.

12. The method of claim 1, wherein the compressed version of the gain map is derived from a gain map having a linear resolution in at least one dimension that is smaller than corresponding linear resolutions of the SDR image and the HDR image.

13. The method according to claim 1, in: The compressed version of the gain map is generated using a first compression scheme optimized for gain mapping; and The compressed version of the image is generated using a second compression scheme optimized for images.

14. The method of claim 1, wherein the computing device uses offset values ​​based on pixel values ​​of the SDR image and / or pixel values ​​of the HDR image when generating the gain map.

15. The method of claim 14, wherein the computing device selects the offset value to optimize storage of the compressed version of the gain map.

16. The method of claim 1, further comprising the computing device: generating an uncompressed version of the gain map from the compressed version of the gain map; determining an error map based on comparing the uncompressed version of the gain map to an original gain map used to generate the compressed version of the gain map; and A compressed version of the error map is stored with the compressed enhanced image.

17. The method according to claim 16, in: The compressed version of the error map is compressed using a lossless compression module; and The compressed version of the gain map is compressed using a lossy compression module.

18. A non-transitory computer-readable storage medium configured to store instructions that, when executed by at least one processor included in a computing device, cause the computing device to manage images by performing steps comprising: generating a compressed version of the image and a compressed version of the gain map from a standard dynamic range (SDR) image of the scene and a high dynamic range (HDR) image of the scene; combining the compressed version of the image with the compressed version of the gain map to form a compressed enhanced image; as well as The compressed enhanced image is stored in a non-volatile storage medium.

19. The non-transitory computer-readable storage medium of claim 18, wherein the steps further comprise, at the second computing device: obtaining the compressed enhanced image; extracting the compressed version of the image and the compressed version of the gain map from the compressed enhanced image; generating an uncompressed version of the image from the compressed version of the image; generating an uncompressed version of the gain map from the compressed version of the gain map; as well as applying the uncompressed version of the gain map to the uncompressed version of the image to generate a second image formatted for display by the second computing device, Wherein the second image and the uncompressed version of the image have brightness values ​​with different dynamic ranges.

20. The non-transitory computer readable storage medium of claim 19, in: The compressed version of the image comprises a compressed version of the SDR image; and The second image comprises a version of the HDR image.

21. The non-transitory computer readable storage medium of claim 19, in: The compressed version of the image comprises a compressed version of the HDR image; and The second image comprises a version of the SDR image.

22. The non-transitory computer-readable storage medium of claim 18, wherein the compressed version of the gain map is generated using a lossy compression module selected for use with gain map compression.

23. The non-transitory computer-readable storage medium of claim 18, wherein the compressed version of the image is generated using a lossy compression module selected for use with image compression.

24. The non-transitory computer-readable storage medium of claim 18, wherein the compressed version of the image and the compressed version of the gain map are generated include: generating a gain map by comparing brightness values ​​of pixels in the HDR image with brightness values ​​of corresponding pixels in the SDR image; generating the compressed version of the image by processing the SDR image or the HDR image using an image compression module; as well as The compressed version of the gain map is generated by processing the gain map using a gain compression module.

25. The non-transitory computer-readable storage medium of claim 18, wherein the compressed version of the image and the compressed version of the gain map are generated include: generating a gain map by comparing brightness values ​​of pixels in the HDR image with brightness values ​​of corresponding pixels in the SDR image; generating the compressed version of the image by jointly processing the SDR image and the HDR image using an image compression module; as well as The compressed version of the gain map is generated by processing the gain map using a gain compression module.

26. The non-transitory computer-readable storage medium of claim 18, wherein the compressed version of the image and the compressed version of the gain map are generated include: generating the compressed version of the image by processing the SDR image using an image compression module; generating a gain map by comparing luminance values ​​of pixels in the HDR image with luminance values ​​of corresponding pixels in the compressed version of the image; as well as The compressed version of the gain map is generated by processing the gain map using a gain compression module.

27. The non-transitory computer-readable storage medium of claim 18, wherein the compressed version of the image and the compressed version of the gain map are generated include: The compressed version of the gain map and the compressed version of the image are jointly generated from the SDR image and the HDR image using a combined gain map generation and compression module.

28. The non-transitory computer-readable storage medium of claim 18, wherein the compressed version of the gain map is derived from a gain map having a linear resolution in each of two dimensions that is the same as a linear resolution of the SDR image and the HDR image.

29. The non-transitory computer-readable storage medium of claim 18, wherein the compressed version of the gain map is derived from a gain map having a linear resolution in at least one dimension that is less than the corresponding linear resolutions of the SDR image and the HDR image.

30. The non-transitory computer readable storage medium of claim 18, in: The compressed version of the gain map is generated using a first compression scheme optimized for gain mapping; and The compressed version of the image is generated using a second compression scheme optimized for images.

31. The non-transitory computer-readable storage medium of claim 18, wherein the computing device uses offset values ​​based on pixel values ​​of the SDR image and / or pixel values ​​of the HDR image when generating the gain map.

32. The non-transitory computer-readable storage medium of claim 31, wherein the computing device selects the offset value to optimize storage of the compressed version of the gain map.

33. The non-transitory computer readable storage medium of claim 18, wherein the steps further comprise the computing device: generating an uncompressed version of the gain map from the compressed version of the gain map; determining an error map based on comparing the uncompressed version of the gain map to an original gain map used to generate the compressed version of the gain map; and A compressed version of the error map is stored with the compressed enhanced image.

34. The non-transitory computer readable storage medium of claim 33, in: The compressed version of the error map is compressed using a lossless compression module; and The compressed version of the gain map is compressed using a lossy compression module.

35. A computing device configured to manage an image, the computing device include: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the computing device to perform steps comprising: generating a compressed version of the image and a compressed version of the gain map from a standard dynamic range (SDR) image of the scene and a high dynamic range (HDR) image of the scene; combining the compressed version of the image with the compressed version of the gain map to form a compressed enhanced image; as well as The compressed enhanced image is stored in a non-volatile storage medium.

36. The computing device of claim 35, wherein the steps further comprise, at the second computing device: obtaining the compressed enhanced image; extracting the compressed version of the image and the compressed version of the gain map from the compressed enhanced image; generating an uncompressed version of the image from the compressed version of the image; generating an uncompressed version of the gain map from the compressed version of the gain map; as well as applying the uncompressed version of the gain map to the uncompressed version of the image to generate a second image formatted for display by the second computing device, Wherein the second image and the uncompressed version of the image have brightness values ​​with different dynamic ranges.

37. The computing device of claim 36, in: The compressed version of the image comprises a compressed version of the SDR image; and The second image comprises a version of the HDR image.

38. The computing device of claim 36, in: The compressed version of the image comprises a compressed version of the HDR image; and The second image comprises a version of the SDR image.

39. The computing device of claim 35, wherein the compressed version of the gain map is generated using a lossy compression module selected for use with gain map compression.

40. The computing device of claim 35, wherein the compressed version of the image is generated using a lossy compression module selected for use with image compression.

41. The computing device of claim 35, wherein generating the compressed version of the image and the compressed version of the gain map include: generating a gain map by comparing brightness values ​​of pixels in the HDR image with brightness values ​​of corresponding pixels in the SDR image; generating the compressed version of the image by processing the SDR image or the HDR image using an image compression module; as well as The compressed version of the gain map is generated by processing the gain map using a gain compression module.

42. The computing device of claim 35, wherein the compressed version of the image and the compressed version of the gain map are generated include: generating a gain map by comparing brightness values ​​of pixels in the HDR image with brightness values ​​of corresponding pixels in the SDR image; generating the compressed version of the image by jointly processing the SDR image and the HDR image using an image compression module; as well as The compressed version of the gain map is generated by processing the gain map using a gain compression module.

43. The computing device of claim 35, wherein generating the compressed version of the image and the compressed version of the gain map include: generating the compressed version of the image by processing the SDR image using an image compression module; generating a gain map by comparing luminance values ​​of pixels in the HDR image with luminance values ​​of corresponding pixels in the compressed version of the image; as well as The compressed version of the gain map is generated by processing the gain map using a gain compression module.

44. The computing device of claim 35, wherein generating the compressed version of the image and the compressed version of the gain map include: The compressed version of the gain map and the compressed version of the image are jointly generated from the SDR image and the HDR image using a combined gain map generation and compression module.

45. The computing device of claim 35, wherein the compressed version of the gain map is derived from a gain map having a linear resolution in each of two dimensions that is the same as a linear resolution of the SDR image and the HDR image.

46. ​​The computing device of claim 35, wherein the compressed version of the gain map is derived from a gain map having a linear resolution in at least one dimension that is less than a corresponding linear resolution of the SDR image and the HDR image.

47. The computing device of claim 35, in: The compressed version of the gain map is generated using a first compression scheme optimized for gain mapping; and The compressed version of the image is generated using a second compression scheme optimized for images.

48. The computing device of claim 35, wherein the computing device uses offset values ​​based on pixel values ​​of the SDR image and / or pixel values ​​of the HDR image when generating the gain map.

49. The computing device of claim 48, wherein the computing device selects the offset value to optimize storage of the compressed version of the gain map.

50. The computing device of claim 35, wherein the steps further comprise the computing device: generating an uncompressed version of the gain map from the compressed version of the gain map; determining an error map based on comparing the uncompressed version of the gain map to an original gain map used to generate the compressed version of the gain map; and A compressed version of the error map is stored with the compressed enhanced image.

51. The computing device of claim 50, in: The compressed version of the error map is compressed using a lossless compression module; and The compressed version of the gain map is compressed using a lossy compression module.

52. A computing device configured to manage an image, the computing device include: means for generating a compressed version of the image and a compressed version of the gain map from a standard dynamic range (SDR) image of a scene and a high dynamic range (HDR) image of the scene; means for combining the compressed version of the image with the compressed version of the gain map to form a compressed enhanced image; and Means for storing the compressed enhanced image in a non-volatile storage medium.

53. The computing device of claim 52, wherein the second computing device comprises means for: obtaining the compressed enhanced image; extracting the compressed version of the image and the compressed version of the gain map from the compressed enhanced image; generating an uncompressed version of the image from the compressed version of the image; generating an uncompressed version of the gain map from the compressed version of the gain map; as well as applying the uncompressed version of the gain map to the uncompressed version of the image to generate a second image formatted for display by the second computing device, Wherein the second image and the uncompressed version of the image have brightness values ​​with different dynamic ranges.

54. The computing device of claim 53, in: The compressed version of the image comprises a compressed version of the SDR image; and The second image comprises a version of the HDR image.

55. The computing device of claim 53, in: The compressed version of the image comprises a compressed version of the HDR image; and The second image comprises a version of the SDR image.

56. The computing device of claim 52, wherein the compressed version of the gain map is generated using a lossy compression module selected for use with gain map compression.

57. The computing device of claim 52, wherein the compressed version of the image is generated using a lossy compression module selected for use with image compression.

58. The computing device of claim 52, wherein the compressed version of the image and the compressed version of the gain map are generated include: generating a gain map by comparing brightness values ​​of pixels in the HDR image with brightness values ​​of corresponding pixels in the SDR image; generating the compressed version of the image by processing the SDR image or the HDR image using an image compression module; as well as The compressed version of the gain map is generated by processing the gain map using a gain compression module.

59. The computing device of claim 52, wherein the compressed version of the image and the compressed version of the gain map are generated include: generating a gain map by comparing brightness values ​​of pixels in the HDR image with brightness values ​​of corresponding pixels in the SDR image; generating the compressed version of the image by jointly processing the SDR image and the HDR image using an image compression module; as well as The compressed version of the gain map is generated by processing the gain map using a gain compression module.

60. The computing device of claim 52, wherein the compressed version of the image and the compressed version of the gain map are generated include: generating the compressed version of the image by processing the SDR image using an image compression module; generating a gain map by comparing luminance values ​​of pixels in the HDR image with luminance values ​​of corresponding pixels in the compressed version of the image; as well as The compressed version of the gain map is generated by processing the gain map using a gain compression module.

61. The computing device of claim 52, wherein the compressed version of the image and the compressed version of the gain map are generated include: The compressed version of the gain map and the compressed version of the image are jointly generated from the SDR image and the HDR image using a combined gain map generation and compression module.

62. The computing device of claim 52, wherein the compressed version of the gain map is derived from a gain map having a linear resolution in each of two dimensions that is the same as a linear resolution of the SDR image and the HDR image.

63. The computing device of claim 52, wherein the compressed version of the gain map is derived from a gain map having a linear resolution in at least one dimension that is less than the corresponding linear resolutions of the SDR image and the HDR image.

64. The computing device of claim 52, in: The compressed version of the gain map is generated using a first compression scheme optimized for gain mapping; and The compressed version of the image is generated using a second compression scheme optimized for images.

65. The computing device of claim 52, wherein the computing device uses offset values ​​based on pixel values ​​of the SDR image and / or pixel values ​​of the HDR image when generating the gain map.

66. The computing device of claim 65, wherein the computing device selects the offset value to optimize storage of the compressed version of the gain map.

67. The computing device of claim 52, further comprising means for: generating an uncompressed version of the gain map from the compressed version of the gain map; determining an error map based on comparing the uncompressed version of the gain map to an original gain map used to generate the compressed version of the gain map; and A compressed version of the error map is stored with the compressed enhanced image.

68. The computing device of claim 67, in: The compressed version of the error map is compressed using a lossless compression module; and The compressed version of the gain map is compressed using a lossy compression module.

Images