Image-specific global tone curve and residual gain map generation based on multiple images

Abstract

Various techniques are disclosed for generating image-specific global tone curves and residual gain maps based on multiple images. A gain map is generated based on a comparison of the first image and the second image. A gain mapping scatter diagram is constructed based on the gain mapping values and channel values of the first image or the second image. An image-specific global tone curve is determined by curve-fitting the gain-mapped scatter diagram. A residual gain map is generated based on a comparison of the gain map to a reconstructed gain map derived from the image-specific global tone curve. An approximation of the second image can be constructed using the image-specific global tone curve and the first image. A copy of the second image can be constructed using the image-specific global tone curve, the residual gain map, and the first image.

Description

Technical Field

[0001] The embodiments described herein illustrate techniques for generating image-specific global tone curves and residual gain maps based on multiple images. Specifically, gain maps can be generated based on a comparison of a first image and a second image. A gain map scatter plot can be constructed based on gain map values ​​and brightness (channel) values ​​of the first or second image. Image-specific global tone curves can be determined by curve fitting to the gain map scatter plot, and residual gain maps can be generated based on a comparison of the gain map with a reconstructed gain map derived from the image-specific global tone curve. Background Technology

[0002] The dynamic range of an image refers to the range of pixel values ​​(often referred to as "luminance") between the brightest and darkest parts of the image. It's important to note that conventional image sensors can only capture a limited range of luminance in a single exposure of a scene, at least relative to the luminance the human eye can perceive from the same scene. This limited range is commonly referred to as standard dynamic range (SDR) in the field of digital photography.

[0003] Despite the aforementioned limitations of image sensors, advancements in photographic techniques have enabled the capture of a wider range of light (referred to herein as High Dynamic Range (HDR)). This is achieved by (1) capturing multiple “bracketed exposure” images, i.e., images with different exposure times; and then (2) combining the bracketed exposure images into a single image that incorporates different aspects of the different exposures. At this point, a single HDR image has a wider dynamic range of brightness compared to the brightness that can be captured in each of the individual exposures. This makes HDR images superior to SDR images in several respects.

[0004] Thanks to advancements in design and manufacturing technologies, display devices capable of displaying HDR images (in their true form) are becoming increasingly readily available. However, most display devices currently in use (and continuing to be manufactured) are only capable of displaying SDR images. Therefore, devices with SDR-limited displays that receive HDR images must perform various tasks to convert (i.e., downgrade) the HDR image to its SDR equivalent. Conversely, devices with HDR-capable displays that receive SDR images can attempt to perform various tasks to convert (i.e., upgrade) the SDR image to its HDR equivalent.

[0005] Using a fixed global tone curve to convert an image from an HDR image to an SDR image or from an SDR image to an HDR image can generate an alternative version of the image that differs from the intention of the original image's creator or initiator.

[0006] Therefore, there is a need for a technology that enables images to be transformed efficiently and accurately between different versions of an image while more accurately maintaining the image creator's intent. Summary of the Invention

[0007] The representative embodiments described herein illustrate techniques for generating image-specific global tone curves and residual gain maps based on multiple images. A gain map scatter plot can be constructed based on gain map values ​​and brightness (channel) values ​​of a first or second image, where the gain map represents a mapping between pixels (or more generally, cell regions) in the first image and pixels (cell regions) in the second image. In some embodiments, the gain map may be a forward mapping from the first image to the second image. In some embodiments, the gain map may be a reverse mapping from the second image to the first image. The image-specific global tone curve can be determined by curve fitting the gain map scatter plot, and the residual gain map can be generated based on a comparison of the gain map with a reconstructed gain map derived from the image-specific global tone curve.

[0008] An exemplary embodiment illustrates a method for generating an image-specific global tone curve from a first image and a second image. The method may include: (1) obtaining a gain map based on a comparison between pixels (cell regions) of the first image and corresponding pixels (cell regions) of the second image, the gain map including a set of gain values ​​for the pixels (cell regions) of the first and second images; (2) generating a scatter plot including a set of points, each point including a gain value and a brightness value, wherein the brightness value may be generated from corresponding channel values ​​for the pixels (cell regions) of the first image, and wherein the gain value represents a mapping between the pixels (cell regions) of the first image and the corresponding pixels (cell regions) of the second image; (3) determining an image-specific global tone curve based on curve fitting of the scatter plot using a cost function; and (4) storing the image-specific global tone curve together with the first image. In some embodiments, the method further includes: (5) generating a reconstructed gain map from the image-specific global tone curve; and (6) determining a residual gain map based on a comparison between pixels (cell regions) of the reconstructed gain map and corresponding pixels (cell regions) of the gain map.

[0009] In some implementations, the first image and the second image have different channel value dynamic ranges (or different corresponding brightness value ranges). In some implementations, an image-specific global tone curve and the first image can be used to construct an approximation of the second image. In some implementations, an image-specific global tone curve, a residual gain map, and the first image can be used to construct an exact copy of the second image. In some implementations, the image-specific global tone curve comprises multiple consecutive segments, each corresponding to a polynomial function such as a cubic spline. In some implementations, curve fitting constrains the image-specific global tone curve to a monotonically increasing function. In some implementations, an image-specific global tone curve and the second image can be used to construct an approximation of the first image. In some implementations, an image-specific global tone curve, a residual gain map, and the second image can be used to construct an exact copy of the first image.

[0010] Other embodiments include a non-transitory computer-readable storage medium configured to store instructions that, when executed by at least one processor included in a computing device, cause the computing device to perform various steps of any of the methods described above. Further embodiments include a computing device configured to perform various steps of any of the methods described above.

[0011] Other aspects and advantages of the invention will become apparent from the following detailed description, taken in conjunction with the accompanying drawings which illustrate the principles of the described embodiments by way of example. Attached Figure Description

[0012] This disclosure will be more readily understood by taking into account the following detailed description in conjunction with the accompanying drawings, wherein similar reference numerals denote similar structural elements.

[0013] Figure 1 An overview of computing devices that can be configured to perform the various technologies described herein, according to some implementation schemes, is illustrated.

[0014] Figure 2A , Figure 2B , Figure 2C , Figure 2D and Figure 2E A series of conceptual diagrams illustrating techniques for generating image-specific global tone curve mappings based on a first (baseline) image and a second (alternative) image, according to some implementation schemes.

[0015] Figure 2F A diagram illustrating techniques for generating residual gain maps according to some implementation schemes is shown.

[0016] Figure 3A and Figure 3BExamples of functional relationships between images, gain mappings, scatter plots, and global tone curves according to some implementation schemes are shown.

[0017] Figure 4 A flowchart illustrating a representative method for generating a specific global tone curve for an image based on two images, according to some implementation schemes, is shown.

[0018] Figure 5 Detailed views of computing devices that can be used to implement the various technologies described herein, according to some implementation schemes, are illustrated. Detailed Implementation

[0019] This section describes representative applications of the methods and apparatus according to this application. These examples are provided only to add context and aid in understanding the described embodiments. Therefore, it will be apparent to those skilled in the art that the described embodiments can be practiced without some or all of these specific details. In other instances, well-known processing steps have not been described in detail to avoid unnecessarily obscuring the described embodiments. Other applications are possible, such that the following examples should not be considered limiting.

[0020] In the following detailed description, reference is made to the accompanying drawings, which form part of this specification, and specific embodiments according to the described embodiments are illustrated by way of example in the drawings. Although these embodiments are described in sufficient detail to enable those skilled in the art to practice the described embodiments, it should be understood that these examples are not limiting, and other embodiments may be used, and changes may be made without departing from the spirit and scope of the described embodiments.

[0021] The representative implementation described herein discloses a technique for generating gain maps based on acquired images. Specifically, a gain map can be generated by comparing a first image with a second image. The gain map can then be embedded into the first image, enabling efficient reproduction of the second image using the first image and the gain map. The following description, in conjunction with… Figure 1 , Figures 2A to 2F , Figure 3A , Figure 3B , Figure 4 and Figure 5 A more detailed description of these technologies is provided.

[0022] Figure 1 An overview 100 of a computing device 102 is illustrated, which may be configured to perform the various techniques described herein. For example... Figure 1 As shown, the computing device 102 may include a processor 104, volatile memory 106, and non-volatile memory 124. It should be noted that... Figure 5More detailed exploded diagrams of example hardware components that may be included in computing device 102 are illustrated below, and are for simplification purposes only. Figure 1 These components are omitted in the examples. For example, computing device 102 may include additional non-volatile memory (e.g., solid-state drives, hard disk drives, etc.), other processors (e.g., multi-core central processing units (CPUs), graphics processing units (GPUs), etc.). According to some implementations, an operating system (OS) ( Figure 1 (Not illustrated herein) can be loaded into volatile memory 106, where the OS executable collectively enables various applications to implement the various techniques described herein. For example, these applications may include an image analyzer 110 (and its internal components), a gain map generator 120 (and its internal components), an image-specific global tone curve generator 122, a residual gain map generator 126, one or more compressors ( Figure 1 (not shown in the example), etc.

[0023] like Figure 1 As shown, the volatile memory 106 can be configured to receive the multi-channel image 108. The multi-channel image 108 can be, for example, a digital imaging unit configured to capture and process digital images (…). Figure 1 (Not illustrated herein) is provided. According to some embodiments, the multi-channel image 108 may consist of a set of pixels, wherein each pixel in the set of pixels includes a group of subpixels (e.g., red subpixels, green subpixels, blue subpixels, alpha subpixels, etc.). It should be noted that the term "subpixel" as used herein may be synonymous with the term "channel". It should also be noted that the multi-channel image 108 may have different resolutions, layouts, bit depths, etc., without departing from the scope of this disclosure.

[0024] According to some implementations, a first multi-channel image 108 may represent a standard dynamic range (SDR) image, which constitutes a single exposure of a scene collected and processed by the digital imaging unit. A second multi-channel image 108 may also represent a high dynamic range (HDR) image, which constitutes multiple exposures of a scene collected and processed by the digital imaging unit. To generate an HDR image, the digital imaging unit may capture a scene with bracketed exposures for different exposure values ​​(e.g., three exposure values ​​that may be referred to as "EV0", "EV-", and "EV+"). Generally, the EV0 image corresponds to a normal / ideal exposure of the scene (typically captured using the automatic exposure settings of the digital imaging unit); the EV- image corresponds to an underexposed image of the scene (e.g., four times darker than EV0); and the EV+ image corresponds to an overexposed image of the scene (e.g., four times brighter than EV0). The digital imaging unit may combine different exposures to produce a resulting image that incorporates a wider range of brightness relative to the SDR image. It should be noted that the multi-channel image 108 discussed herein is not limited to SDR / HDR images. Conversely, without departing from the scope of this disclosure, the multi-channel image 108 may represent any form of digital image (e.g., scanned image, computer-generated image, etc.).

[0025] like Figure 1 As shown, the multi-channel image 108 can (optionally) be provided to the image analyzer 110. According to some embodiments, the image analyzer 110 may include various components configured to process / modify the multi-channel image 108 and generate information based on these multi-channel images. For example, the image analyzer 110 may include a tone mapping unit 112 (e.g., configured to perform global / local tone mapping operations, inverse tone mapping operations, etc.), a noise reduction unit 114 (e.g., configured to reduce global / local noise in the multi-channel image), a color correction unit 116 (e.g., configured to perform global / local color correction in the multi-channel image), and a sharpening unit 118 (e.g., configured to perform global / local sharpening correction in the multi-channel image). It should be noted that, without departing from the scope of this disclosure, the image analyzer 110 is not limited to the aforementioned processing units, and the image analyzer 110 may incorporate any number of processing units configured to perform any processing / modification on the multi-channel image 108.

[0026] like Figure 1As shown, after processing by image analyzer 110, multichannel image 108 can be provided to gain map generator 120. However, it should be noted that, without departing from the scope of this disclosure, multichannel image 108 can bypass image analyzer 110 and be provided to gain map generator 120 if necessary. It should also be noted that, without departing from the scope of this disclosure, multichannel image 108 can bypass one or more processing units in the processing unit of image analyzer 110. For example, two given multichannel images can be passed through tone mapping unit 112 to receive local tone mapping modifications and then bypass the remaining processing units in image analyzer 110. In this respect, two multichannel images that have undergone local tone mapping operations can be used to generate gain map 123 reflecting the performed local tone mapping operations.

[0027] In any case, and as described in more detail herein, the gain map generator 120 may generate a gain map 123 based on the two multichannel images 108 upon receipt. According to some embodiments, the gain map 123 may be used to generate a scatter plot from which an image-specific global tone curve generator 122 may derive an image-specific global tone curve (isGTC) 125. In some embodiments, the scatter plot may be parameterized using: i) gain values ​​from cell regions (individual pixels or groups of pixels) of the gain map 123, and ii) channel values ​​from the corresponding cell regions of one of the two multichannel images 108 (e.g., the first (baseline) multichannel image). Each point (dot) in the scatter plot is associated with a specific gain value and a specific channel value (e.g., a luminance channel value). Multiple cell regions of the gain map and corresponding multiple cell regions of the multichannel images may have the same specific gain value and specific channel value. Therefore, each point in the scatter plot may include a count (or normalized) of cell regions having that specific gain value and specific channel value. The cell region of the multi-channel image 108 can be an individual pixel of the multi-channel image 108, or a group representing consecutive pixels of the multi-channel image 108, such as a group of four adjacent pixels.

[0028] Therefore, the gain mapping scatter plot can be characterized by counting the pairs of channel values ​​and gain values ​​appearing in the cell regions. Alternatively, the scatter plot can be based on a graph of the channel values ​​for the corresponding cell regions from each of the two multichannel images 108, for example, a graph of the channel values ​​for the cell regions of the first multichannel image 108 relative to the channel values ​​for the corresponding cell regions of the second multichannel image 108. isGTC 125 can be derived by determining a curve fitted (e.g., based on an optimization cost function) to the scatter plot. In some embodiments, isGTC 125 can consist of consecutive segmented pieces, each segment being characterized by a polynomial function of, for example, cubic splines. According to some embodiments, isGTC 125 can be used to derive an approximation of the second multichannel image 108 from the first multichannel image 108. The residual gain mapping generator 126 can determine the residual gain mapping 127 by comparing the reconstructed gain mapping derived from isGTC 125 with a previously determined gain mapping 123. A copy of the second multichannel image 108 can be derived from the first multichannel image 108 using isGTC 125 and residual gain mapping 127.

[0029] therefore, Figure 1 A high-level overview of the various hardware / software architectures that can be implemented by computing device 102 to perform the various technologies described herein is provided. These will now be combined with... Figures 2A to 2F and Figure 3 to Figure 4 To provide a more detailed breakdown of these technologies.

[0030] Figures 2A to 2F A series of conceptual diagrams illustrating techniques for generating image-specific global tone curves 125 and residual gain maps 264 based on a first (baseline) image 211 and a second (alternate) image 221, according to some embodiments. Figure 2A , Figure 2B and Figure 2C The steps outlined herein can be performed "offline," meaning that the step for generating gain mapping 123 can be performed separately from the subsequent step for generating an image-specific global tone curve, which can be generated using the provided gain mapping associated with a pair of images. For example... Figure 2A As shown, the first offline step may involve computing device 102 accessing a first (baseline) image 211 composed of pixels 212 (each denoted as "P"). More generally, the first (baseline) image 211 may be composed of unit regions comprising one or more consecutive pixels. For example, a unit region may comprise four adjacent square pixels forming a larger square, or it may be a composite "larger" pixel derived from a plurality of adjacent "smaller" pixels. Figure 2AAs shown, pixels 212 can be arranged according to a row / column layout, where the subscript "P" (e.g., "1,1") of each pixel 212 indicates the position of pixel 212 according to the row and column. The pixels 212 of the first (baseline) image 211 can be arranged in an equal number of rows and columns, such that the first (baseline) image 211 is a square image. However, it should be understood that the techniques described herein can be applied to multi-channel images 108 with different layouts (e.g., disproportionate row / column counts), i.e., for the first (baseline) image 211, the number of rows "R" may differ from the number of columns "C". Figure 2A As shown, each pixel 212 may consist of three sub-pixels 214—a red sub-pixel 214 (denoted as "R"), a green sub-pixel 214 (denoted as "G"), and a blue sub-pixel 214 (denoted as "B"). More generally, the sub-pixels 214 can be considered as component values ​​of a unit region. However, it should be noted that each pixel 212 may consist of any number of sub-pixels 214 without departing from the scope of this disclosure.

[0031] Figure 2B This illustrates a second offline step involving computing device 102 accessing a second (alternative) image 221. For example... Figure 2B As shown, the second (alternative) image 221 is composed of... Figure 2A The first (baseline) image 211 illustrated herein is composed of pixels 222 or more generally unit regions (and sub-pixels 224 or more generally unit region components) similar to pixels 212 (and sub-pixels 214). In some embodiments, the first (baseline) image is a standard dynamic range (SDR) image, and the second (alternative) image 221 is a high dynamic range (HDR) image. In some embodiments, the first (baseline) image is an HDR image, and the second (alternative) image 221 is an SDR image. In some embodiments, the second (alternative) image 221 is a modified (e.g., photo-edited) version of the first (baseline) image 211. In some embodiments, the first (baseline) image 211 is a single-exposure capture derived from multiple captures of the same scene from different exposures captured by the second (alternative) image 221, such that the first (baseline) image 211 and the second (alternative) image 221 are substantially correlated with each other. For example, if a second (alternative) image 221 is generated using bracketed exposure images at EV-, EV0, and EV+ settings as discussed herein, the first (baseline) image 211 may be based on an EV0 exposure (e.g., before the EV0 exposure is merged with EV- and EV+ exposures to generate the second (alternative) image 221). The pixels of the first (baseline) image 211 and the second (alternative) image 221 may differ in terms of captured variations in channel values ​​(such as brightness levels) and / or pixel color values.

[0032] Figure 2CA third offline step is illustrated, which involves computing device 102 comparing pixels (cell regions) of a first (baseline) image 211 with pixels (cell regions) of a second (alternate) image 221. Figure 2C The gain map 231 (illustrated as comparison 234) is generated (consisting of pixels 232 or more generally, unit regions). In some embodiments, the channel value (or composite channel value for each unit region) for the second (alternate) image 221 is divided by the channel value (or composite channel value for the unit region) of the corresponding pixel (or unit region) of the first (baseline) image 211 to produce a quotient. The corresponding quotient can then be assigned to the value of the corresponding pixel 232 (unit region) of the gain map 231. For example, if the first (baseline) image 211 is represented as "P"... 1,1 The pixel of the second (alternative) image 221 has a value of "5", and the pixel of the second (alternative) image 221 is represented as "P". 1,1 If a pixel has a value of "1", then the quotient will be "0.2" and will be assigned to gain map 231, which is represented as "P". 1,1 The value of the pixel is "". In this way, the second (alternative) image 221 is represented as "P". 1,1 The pixels of “P” can be represented by the first (baseline) image 211 as “P”. 1,1 The pixel with the value "5" multiplied by the gain mapping 231 is represented as "P". 1,1 The pixels with a value of "0.2" are used to reproduce the image. Specifically, the multiplication will produce a product of "1", which, together with the second (alternative) image 221, is represented as "P". 1,1 The pixel value of "1" matches. Therefore, gain mapping 231 and the first (baseline) image 211 can be used to generate a copy of the second (alternate) image 221. Alternatively, the channel values ​​of the pixels used for the first (baseline) image 211 (or the composite channel values ​​used for the cell region) can be divided by the channel values ​​of the second (alternate) image 221 (or the composite channel values ​​used for the cell region) to create an inverse gain mapping.

[0033] In short, it is important to note that, although Figure 2CThe comparisons illustrated (and described herein) are pixel-level comparisons, but embodiments are not limited thereto. Instead, without departing from the scope of this disclosure, pixels of an image can be compared to each other at any granularity. For example, subpixels of a first (baseline) image 211 can be compared to subpixels of a second (alternative) image 221 (as an alternative or supplement to pixel-level comparisons), allowing multiple gain maps (e.g., corresponding gain maps for each channel of the image) to be generated under different comparison methods. In some embodiments, a luminance gain map can be determined by comparing a luminance value representing the brightness level of each pixel and a combination of subpixel values ​​for different color channels (or values ​​obtainable from a monochrome image) between the first (baseline) image and the second (alternative) image.

[0034] Additionally, it should be noted that various optimizations can be employed when generating the gain map without departing from the scope of this disclosure. For example, when two values ​​are identical, the comparison operation can be skipped, and a single bit value (e.g., "0") can be assigned to the corresponding value in the gain map to minimize the size of the gain map (i.e., storage requirements). Furthermore, the resolution of the gain map can be smaller than the resolution of the images compared to generate the gain map. For example, an approximation of every four pixels in the first image can be compared with an approximation of every four corresponding pixels in the second image to generate a gain map with a resolution one-quarter that of the first and second images. This method will significantly reduce the size of the gain map, but will reduce the overall accuracy (or vice versa) of reproducing the first image from the second image and the gain map. Additionally, the first and second images can be resampled in any conceivable manner before generating the gain map. For example, the first and second images can undergo a local tone mapping operation before generating the gain map.

[0035] Figure 2D A first step for generating a specific global tone curve for an image is illustrated, wherein the first step involves computing device 102 using the values ​​of gain mapping 231 and luminance (channel) values ​​from a first (baseline) image 211 to generate an exemplary scatter plot, such as gain-mapped scatter plot 240. Each pixel (cell region) 212 of the first (baseline) image 211 may have one or more channel values ​​associated with the pixel (cell region) 212. For example, pixel P of the first (baseline) image 211 1,1 (Unit region) 212 can have a red color channel value R 1,1 Green color channel value G 1,1 Blue color channel value B 1,1 Brightness value L 1,1 (Not explicitly shown, but in the case of pixel P) 1,1 When the color channel values ​​of the sub-pixels are combined together, the brightness value can represent pixel P. 1,1(The brightness level). In the following discussion, the brightness values ​​for pixels are used to generate scatter plots, such as gain-mapped scatter plot 240; however, other combinations of monochrome channel values ​​or color channel values ​​for each pixel may also be considered. Multiple pixels of the first (baseline) image 211 may have the same channel value, such as the same (or quantized to the same) brightness value. The gain values ​​of the pixels corresponding to multiple pixels for a given channel value in gain mapping 231 may vary. The brightness level of pixel P of the first (baseline) image 211 i,j The corresponding pixel P' of gain mapping 231 i,j Each pixel pair can be characterized by a specific first (baseline) image channel value 242 and a specific gain value 244. Each pixel pair can be plotted on a gain map scatter plot 240, where the x-axis represents the first (baseline) image channel value 242 and the y-axis represents the gain value 244. Different pixel pairs in different regions of the first (baseline) image 211 and the gain map 231 can have the same first (baseline) image channel value 242 and gain value 244. Therefore, a single point in the gain map scatter plot 240 with a specific first (baseline) image channel value and a specific gain value 244 can represent a count of pixel pairs with those specific pairing values. In some embodiments, the count values ​​associated with the plotted points of the gain map scatter plot 240 can be normalized to a specific range (e.g., from 0 to 1).

[0036] Figure 2D The gain-mapped scatter plot 240 illustrated is based on a pair of gain values ​​244 from gain mapping 231 and first (baseline) image channel values ​​242. Alternative value pairs can also be used to generate the scatter plot. For example, the first (baseline) image channel value 242 can be paired with corresponding channel values ​​from a second (alternate) image 221 to generate a scatter plot based on the first (baseline) image channel value 242 along the x-axis and the second (alternate) image channel value along the y-axis. In some embodiments, the first (baseline) image 211 is an SDR image and the second (alternate) image 221 is an HDR image, and the scatter plot represents (SDR, HDR) channel value pairs. In some embodiments, the first (baseline) image 211 is an HDR image and the second (alternate) image 221 is an SNR image, and the scatter plot represents (HDR, SDR) channel value pairs. In some embodiments, gain-mapped values ​​244 are paired with channel values ​​from the second (alternate) image 221 to generate the scatter plot. Gain mapping 231 can be considered as a function of the first (baseline) image 211 and the second (alternative) image 221, and other functions can also be considered to create alternatives to gain mapping, which can then be used to generate scatter plots.

[0037] It should be noted that, although combined Figure 2DThe techniques described (and others) effectively treat each pixel with equal weight, but implementations are not limited to this. Instead, various analyses, adjustments, etc., can be performed to identify pixels and assign weights to them to influence the generation of the scatter plot itself—and the information ultimately derived from the gain-mapped scatter plot 240 (e.g., image-specific global tone curves, (reconstruction, residual, etc.) gain mapping, etc.). In particular, and according to some implementations, appropriate weights can be assigned to one or more unit regions (pixels, groups of pixels, etc.) of particular interest in the image, such as objects within the image, regions within the image (uniform / uniform in shape), etc. For example, one or more weights can be applied to a group of pixels corresponding to a face in a given image. In another example, one or more weights can be applied to a group of pixels corresponding to a car in a given image. In yet another example, one or more weights can be applied to a group of pixels corresponding to the sky in a given image. In any case, applying such weights to groups of pixels—which can effectively increase (e.g., regions of interest) or decrease (e.g., regions of non-interest) one or more channel values ​​of pixels—can also affect the layout of the gain-mapped scatter plot 240. Therefore—and as described in more detail below—the information generated based on the (result-adjusted) gain mapping scatter plot 240 is also affected / adjusted.

[0038] It should be noted that, without departing from the scope of this disclosure, any method may be used to identify the aforementioned regions and objects of interest within an image, including non-AI methods (e.g., SIFT (Scale Invariant Feature Transform), SURF (Speed-Up Robust Feature Transform), ORB (Oriented Fast and Rotated BRIEF), Harris corner detector, histogram-based matching, template matching, edge detection (e.g., Canny edge detector), color-based segmentation, morphological operations, connectivity component analysis, texture analysis, etc.), AI methods (Haar cascade, convolutional neural networks (CNN), region-based CNNs (R-CNN, Faster R-CNN, Faster R-CNN), YOLO (You Only See Once), SSD (Single-Trigger Multi-Box Detector), masked R-CNN, K-means clustering, oriented gradient histogram (HOG), edge detection (e.g., Canny edge detector), mean-shifted clustering, etc.), and user-input-based methods (e.g., user-provided bounding boxes). Additionally, it should be noted that, without departing from the scope of this disclosure, any of the aforementioned methods may be used to identify the aforementioned weights. Additionally, it should be noted that, without departing from the scope of this disclosure, the corresponding weights can be applied to individual pixels, pixel groups, etc., at any granularity.

[0039] Statistical information for the gain mapping scatter plot 240 can also be generated to determine the curve to be fitted to the gain mapping scatter plot 240. For each specific channel value (e.g., a specific brightness value) in the first (baseline) image 211, there may be multiple individual pixels (cell regions) of the first (baseline) image 211 having that specific channel value. The gain value 244 of the gain mapping 231 corresponding to the multiple individual pixels (cell regions) of the first (baseline) image 211 can span a range of gain values ​​244. Several plotted pixels (cell regions) 246 in the gain mapping scatter plot 240 may have a specific first (baseline) image brightness (channel) value 242 and different gain values ​​244. Therefore, a specific first (baseline) image brightness (channel) value 242 can be associated with a set of gain values ​​244, and statistical information such as mean, standard deviation, percentiles, etc., can be generated for this set. The pixel (cell region) value pairs in gain map 231 can be processed in various ways to generate statistical values ​​that can be used to help generate image-specific global tone curves to fit the gain map scatter plot 240 data. According to some embodiments, in some implementations, different statistical probability functions for each first (baseline) image brightness (channel) value 242 can be derived and used to characterize the gain map scatter plot 240, for example, to help derive the curve to be fitted to the gain map scatter plot 240. In some implementations, one or more statistical histograms and / or probability density functions are determined based on the gain map scatter plot 240 data and can be used to estimate the image-specific global tone curve 250 to be fitted to the gain map scatter plot 240 data.

[0040] Figure 2EThe second step is illustrated, in which an image-specific global tone curve 250 is generated from the plotted pixels (cell regions) 246 of the gain-mapped scatter plot 240. The image-specific global tone curve 250 can be determined using one or more model fitting methods such as: i) polynomial fitting; ii) parametric fitting with generalized logistic functions, sigmoids, hyperbolic tangents, etc.; iii) multi-level pyramids; iv) piecewise linear fitting, piecewise cubic splines, pchip, makima; v) lookup tables (LUTs) where the midpoint is set to a certain percentile and linear interpolation is performed between the midpoints; vi) LUTs derived via curve fitting; and / or vii) multiple stages: a coarse LUT, followed by cubic splines / pchip / makima, followed by a finer LUT. Various cost functions can be used to determine the optimal curve fit, such as based on: i) square or square root distance, ii) structural similarity index measure (SSIM), iii) variance inflation factor (VIF), and / or iv) statistical measures (mean, median, maximum, percentile, etc.). In some implementations, one or more heuristic rules are applied to determine the optimal curve fit, wherein the heuristic rules are selected to improve the image quality of an image that can be generated using an image-specific global tone curve 250. Exemplary heuristic rules may include: i) limiting the negative slope value of the image-specific global tone curve 250 for higher channel values ​​(e.g., higher brightness or luminance values) and / or after the peak of the image-specific global tone curve 250 occurs; ii) ensuring the continuity and / or smoothness of the image-specific global tone curve 250; iii) detecting and compensating for the multimodal distribution of the data; iv) adjusting the weighting factors for the multimodal data distribution and / or for data with large variance values; and v) adjusting pixels (cell regions) with null values.

[0041] Image-specific global tone curve 250 effectively combines local tone curve information for different regions of the first (baseline) image 211 and the second (alternate) image 221. Different optimization algorithms combining various rules (including heuristic rules in some cases) can result in different image-specific global tone curves 250 with varying characteristics. For example, one approach may produce an image-specific global tone curve 250 that more closely matches the brighter regions (higher brightness values) of the image, while another approach may produce an image-specific global tone curve 250 that more closely matches the darker regions (lower brightness values) of the image. In some implementations, options may be implemented for adjusting the algorithm to determine the image-specific global tone curve 250, such as allowing the generation of an image-specific global tone curve 250 that more closely matches the desired result (e.g., biased towards a brighter image or a darker image).

[0042] In some implementations, options for selecting image type (e.g., indoor portrait versus outdoor landscape) can be used to determine specific parameters to be used in model fitting to generate image-specific global tone curves 250. In some cases, multiple image-specific global tone curves 250 may be provided, each with different characteristics to allow for subsequent selection based on user-defined preferences. Generally, image-specific global tone curves 250 contain less information than the gain map 231 from which they are derived, and may require less storage. By wisely selecting optimization algorithms, images generated using the limited information of image-specific global tone curves 250, rather than the complete information in the gain map 231, can be perceptibly minimally different. Therefore, image-specific global tone curves 250 provide a more efficient representation of the differences between two images. In some cases, image-specific global tone curves 250 may be represented by a set of coefficients and / or parameter values ​​for one or more functions. In some cases, image-specific global tone curves 250 may be represented as a lookup table (LUT) of values.

[0043] Figure 2FThe third step is illustrated, in which a residual gain map 264 is generated using an image-specific global tone curve 250 and a gain map 231 derived from a comparison of a first (baseline) image 211 with a second (alternate) image 221. The image-specific global tone curve 250 can be used to determine a reconstructed gain map 260 that includes gain values ​​for each pixel (cell region) 262 across images 211 / 221. Because generating the image-specific global tone curve 250 is not a lossless process (i.e., local information is lost during its generation), the reconstructed gain map 260 will differ from the gain map 231 directly derived from images 211 / 221. The reconstructed gain map 260 and the gain map 231 can be compared to each other to determine the residual gain map 264, which includes values ​​for each pixel (cell region) 266. For example, the difference between each pixel (cell region) 262 of the reconstructed gain map 260 and the corresponding pixel (cell region) 232 of the gain map 231 can be determined to calculate the residual gain map 264 value for the pixel (cell region) 266 of the residual gain map 264. The gain map 231 can be used to generate an exact copy of the second (alternate) image 221 from the first (baseline) image 211. (The gain map 231 can also be used to generate an exact copy of the first (baseline) image 211 from the second (alternate) image 221 by appropriately applying an inverse gain value to generate the second (alternate) image 221 from the first (baseline) image 211.) The image-specific global tone curve 250 can be used, for example, via the reconstructed gain map 260 to generate a variant of the second (alternate) image 221 from the first (baseline) image 211. The residual gain map 264 captures the difference between the "full information" gain map 231 and the "reduced information" reconstructed gain map 260 generated from the image-specific global tone curve 250. In some embodiments, the residual gain map 264 may be stored together with the image-specific global tone curve 250 to allow for the reproduction of an exact copy of the second (alternate) image 221 from the first (baseline) image 211. In some embodiments, the residual gain map 264 and the image-specific global tone curve 250 may be used to reproduce an exact copy of the first (baseline) image 211 from the second (alternate) image 221, thereby appropriately inverting the gain value when the residual gain map and the image-specific global tone curve are applied.

[0044] In short, it is important to note that, although Figures 2D to 2FThe generation of a single image-specific global tone curve 250 for a given image has been discussed, but the implementation is not limited thereto. Instead, any number of image-specific global tone curves 250 at any granularity can be generated for a given image without departing from the scope of this disclosure. For example, two or more regions of a given image—such as objects, areas (uniform / uniform in shape), etc.—can be identified (e.g., using non-AI, AI, user input, etc. methods discussed herein), and then a combination can be implemented for each of the two or more regions. Figures 2D to 2F The described technique generates a corresponding image-specific global tone curve 250 for that region. Each image-specific global tone curve 250 can then be paired with information identifying a corresponding region of the image that corresponds to the image-specific global tone curve 250, and can then be stored together with the image. In this way, when the image is being displayed, the image-specific global tone curve 250 can enable appropriate adjustments to the region where necessary.

[0045] Additionally, it should be noted that enhanced features can be provided when a given image is associated with two or more image-specific global tone curves 250, such as when multiple image-specific global tone curves 250 are generated for different regions of the image, or when two or more versions of image-specific global tone curves 250 are generated for a given region of the image, etc. For example, user interfaces can be provided that allow any image-specific global tone curve 250 to be applied to the corresponding region of these image-specific global tone curves within the image. Furthermore, without departing from the scope of this disclosure, the user interface can enable any image-specific global tone curve 250 to be adjusted at any granularity based on information of any quantity, type, form, etc. This flexibility can advantageously allow end users to view the effect of the image-specific global tone curves 250 when they are applied to an image, the effect of any adjustment to the image-specific global tone curves 250 when these image-specific global tone curves are applied to an image, etc.

[0046] Figure 3AA diagram 300 illustrates the functional relationships between the various components discussed above. Gain mapping 231 can be generated as the output of a first function F1, which takes a first (baseline) image 211 and a second (alternate) image 221 as input. Gain mapping 231 captures a comparison between the first (baseline) image 211 and the second (alternate) image 221. The second (alternate) image 221 can be regenerated as the output of a second function F2, which takes the first (baseline) image 211 and gain mapping 231 as input. A third function F3 can be used to generate a gain mapping scatter plot 240 using the first (baseline) image 211 and gain mapping 231 as input. In some cases, gain mapping scatter plot 240 contains less information than gain mapping 231. A fourth function F4 can be used to generate an image-specific global tone curve 250 from gain mapping scatter plot 240. The fourth function F4 is not a lossless operation, and image-specific global tone curve 250 contains less information than the full gain mapping 231. The fifth function, F5, can be used to generate a reconstructed gain map 260 from the image-specific global tone curve 250. Since the image-specific global tone curve 250 does not capture the complete information of the gain map 231, the reconstructed gain map 260 also does not capture the complete information of the gain map 231. The sixth function, F6, can be used to generate a residual gain map 264 using the reconstructed gain map 260 and the gain map 231 as input. The seventh function, F7, can be used to create a variant version of the second (alternate) image 321 from the first (baseline) image 211 and the image-specific global tone curve 250. In some cases, the reconstructed gain map 260 obtained from the image-specific global tone curve 250 is used to modify the first (baseline) image 211 to generate a variant version of the second (alternate) image 321. The eighth function, F8, can be used to generate an exact copy of the second (alternate) image 221 using the first (baseline) image 211, the image-specific global tone curve 250, and the residual gain map 264 together as input.

[0047] Figure 3B Figure 310 illustrates alternative functional relationships between the various components discussed above. For example, in... Figure 3A In this process, gain mapping 231 can be generated as the output of a first function F1, which takes a first (baseline) image 211 and a second (alternate) image 221 as input. Gain mapping 231 captures the comparison between the first (baseline) image 211 and the second (alternate) image 221. The first (baseline) image 211 can be regenerated as an alternative second function F1. 2A The output of the alternative second function takes the second (alternate) image 221 and the gain map 231 as input. The gain map values ​​can be inverted when applied to cell regions of the second (alternate) image to generate the first (baseline) image. The third alternative function F... 3AIt can be used to generate an alternative gain map scatter plot 240-A using the second (alternate) image 221 and gain map 231 as input. In some cases, the gain map scatter plot 240 contains less information than the gain map 231. A fourth alternative function F 4A This can be used to generate a candidate image-specific global tone curve 250-A from the candidate gain map scatter plot 240-A. Fourth candidate function F 4A It is not a lossless operation, and the candidate image-specific global tone curve 250-A contains less information than the full gain map 231. Fifth candidate function F 5A This can be used to generate a candidate reconstructed gain map 260-A from a candidate image-specific global tone curve 250-A. Since the candidate image-specific global tone curve 250-A does not capture the complete information of gain map 231, the candidate reconstructed gain map 260-A also does not capture the complete information of gain map 231. Sixth candidate function F 6A It can be used to generate an alternative residual gain map 264-A by using alternative reconstructed gain map 260-A and gain map 231 as inputs. Seventh alternative function F 7A This can be used to create another variant version of the second (alternate) image 321-A from the first (baseline) image 211 and the alternative image-specific global tone curve 250-A. In some embodiments, the first (baseline) image 211 is modified using an alternative reconstructed gain map 260-A obtained from the alternative image-specific global tone curve 250-A to generate an additional variant version of the second (alternate) image 321-A. Eighth Alternate Function F 8A This can be used to generate an exact copy of the second (alternate) image 221 using the first (baseline) image 211, the alternative image-specific global tone curve 250-A, and the alternative residual gain map 264-A together as input. When the applicable gain map values ​​can be inverted as needed to map between different first (baseline) images and second (alternate) images, such as when generating a gain map from the first (baseline) image to the second (alternate) image, the inverse value of the gain map can be used to generate the first (baseline) image from the second (alternate) image.

[0048] Figure 4 Examples are given by computing devices (e.g.) Figure 1A flowchart 400 illustrates an exemplary method for a computing device 102 to generate an image-specific global tone curve 250. At step 402, the computing device obtains a gain map including a set of gain values ​​based on a comparison of cell regions in a first image and cell regions in a second image. At step 404, the computing device generates a scatter plot including a set of points, each point including a gain value from the gain map and a brightness (channel) value for the corresponding cell region in the first image (or the second image). At step 406, the computing device determines the image-specific global tone curve based on curve fitting of the scatter plot using a cost function. At step 408, the computing device stores the image-specific global tone curve along with the first image.

[0049] In some embodiments, the method further includes a computing device for: i) generating a reconstructed gain map from an image-specific global tone curve, and ii) determining a residual gain map based on a comparison between a cell region of the reconstructed gain map and a corresponding cell region of the gain map. In some embodiments, the second image can be reproduced using a combination of the first image, the image-specific global tone curve, and the residual gain map. In some embodiments, the residual gain map includes the difference between the gain value of a cell region of the reconstructed gain map and the gain value of a corresponding cell region of the gain map. In some embodiments, the first image is a standard dynamic range (SDR) image, and the second image is a high dynamic range (HDR) image. In some embodiments, the method further includes a computing device for generating a variant second image based on the first image and the image-specific global tone curve, wherein the variant second image differs from the second image in terms of one or more channel values ​​in a corresponding cell region. In some embodiments, the image-specific global tone curve comprises a plurality of consecutive segments, each segment corresponding to a cubic spline. In some embodiments, curve fitting constrains the image-specific global tone curve to a monotonically increasing function. In some implementations, the cost function includes one of the following: a least squares function, a structural similarity index measure (SSIM), or a variance inflation factor (VIF). In some implementations, the channel values ​​of the first image used to generate the scatter plot include the brightness values ​​of the cell regions of the first image. In some implementations, the channel values ​​of the first image used to generate the scatter plot include one or more color values ​​of the cell regions of the first image. In some implementations, each cell region corresponds to a single pixel. In some implementations, each cell region corresponds to multiple consecutive pixels.

[0050] Figure 5 Detailed views of a computing device 500, according to some embodiments, that can be used to implement various technologies described herein. In particular, these detailed views illustrate what may be included in conjunction with... Figure 1 The various components in the described computing device 102. For example... Figure 5As shown, computing device 500 may include a processor 502 representing a microprocessor or controller for controlling the overall operation of computing device 500. Computing device 500 may also include a user input device 508 that allows a user of computing device 500 to interact with it. For example, user input device 508 may take various forms, such as buttons, keypads, dial pads, touchscreens, audio input interfaces, visual / image capture input interfaces, input in the form of sensor data, etc. Additionally, computing device 500 may include a display 510, which may be controlled by processor 502 (e.g., via a graphics component) to display information to a user. Data bus 516 facilitates data transfer between at least storage device 540, processor 502, and controller 513. Controller 513 can be used to interact with and control different devices via device control bus 514. Computing device 500 may also include a network / bus interface 511 coupled to data link 512. In the case of wireless connectivity, network / bus interface 511 may include a wireless transceiver.

[0051] As described above, computing device 500 also includes storage device 540, which may include a single disk or a collection of disks (e.g., a hard disk drive). In some embodiments, storage device 540 may include flash memory, semiconductor (solid-state) memory, etc. Computing device 500 may also include random access memory (RAM) 520 and read-only memory (ROM) 522. ROM 522 may store programs, utilities, or processes that will be executed in a non-volatile manner. RAM 520 may provide volatile data storage and store instructions related to the operation of applications executing on computing device 500, such as image analyzer 110, gain mapping generator 120, image-specific global tone curve generator 122, residual gain mapping generator 126, etc.

[0052] Various aspects, embodiments, specific implementations, or features of the described embodiments may be used individually or in any combination. Various aspects of the described embodiments may be implemented by software, hardware, or a combination of hardware and software. The described embodiments may also be embodied as computer-readable code on a computer-readable medium. A computer-readable medium is any data storage device capable of storing data that can subsequently be read by a computer system. Examples of computer-readable media include read-only memory, random access memory, CD-ROM, DVD, magnetic tape, hard disk drive, solid-state drive, and optical data storage devices. Computer-readable media may also be distributed across a network-coupled computer system, enabling the computer-readable code to be stored and executed in a distributed manner.

[0053] For illustrative purposes, the foregoing description uses specific names to provide a thorough understanding of the described embodiments. However, it will be apparent to those skilled in the art that specific details are not required to practice the described embodiments. Therefore, the foregoing description of specific embodiments is presented for illustrative and descriptive purposes. The foregoing description is not intended to be exhaustive or to limit the described embodiments to the precise forms disclosed. It will be apparent to those skilled in the art that many modifications and variations are possible in light of the teachings above.

Claims

1. A method for generating a specific global tone curve for an image, the method comprising, at a computing device: Gain mapping is obtained based on a comparison between a unit region of a first image and a corresponding unit region of a second image. The gain mapping includes a set of gain values ​​for the unit regions of the first image and the second image. Generate a scatter plot comprising a set of points, each point including a gain value from the gain map and a brightness value for the corresponding cell region of the first image; The specific global tone curve of the image is determined by curve fitting the scatter plot using a cost function. as well as The image-specific global tone curve is stored together with the first image.

2. The method according to claim 1, further comprising: Generate a reconstructed gain map from the specific global tone curve of the image; as well as The residual gain mapping is determined by comparing the cell region of the reconstructed gain mapping with the corresponding cell region of the gain mapping.

3. The method of claim 2, wherein the second image can be reproduced using a combination of the first image, the image-specific global tone curve, and the residual gain mapping.

4. The method of claim 2, wherein the residual gain mapping includes the difference between the gain value of the cell region of the reconstructed gain mapping and the gain value of the corresponding cell region of the gain mapping.

5. The method according to claim 1, wherein: The first image includes a standard dynamic range (SDR) image; and The second image includes a high dynamic range (HDR) image.

6. The method according to claim 1, further comprising: A variant second image is generated based on the first image and the image-specific global tone curve.

7. The method of claim 6, wherein the variant second image differs from the second image in one or more channel values ​​in the corresponding cell region.

8. The method of claim 1, wherein the image-specific global tone curve comprises a plurality of consecutive segments, each segment corresponding to a cubic spline.

9. The method of claim 1, wherein the curve fitting constrains the image-specific global tone curve to a monotonically increasing function.

10. The method of claim 1, wherein the cost function comprises one of the following: a least squares function, a structural similarity index measure (SSIM), or a variance inflation factor (VIF).

11. The method of claim 1, wherein the channel values ​​of the first image used to generate the scatter plot include the brightness values ​​of the cell regions of the first image.

12. The method of claim 1, wherein the channel values ​​of the first image for generating the scatter plot include one or more color values ​​of the cell regions of the first image.

13. The method of claim 1, wherein each unit region corresponds to a single pixel.

14. The method of claim 1, wherein each unit region corresponds to a plurality of consecutive pixels.

15. A non-transitory computer-readable storage medium configured to store instructions, which, when executed by at least one processor included in a computing device, cause the computing device to generate an image-specific global tone curve by performing steps, the steps comprising: Gain mapping is obtained based on a comparison between a unit region of a first image and a corresponding unit region of a second image. The gain mapping includes a set of gain values ​​for the unit regions of the first image and the second image. Generate a scatter plot comprising a set of points, each point including a gain value from the gain map and a brightness value for the corresponding cell region of the first image; The specific global tone curve of the image is determined by curve fitting the scatter plot using a cost function. as well as The image-specific global tone curve is stored together with the first image.

16. The non-transitory computer-readable storage medium of claim 15, wherein the step further comprises: Generate a reconstructed gain map from the specific global tone curve of the image; as well as The residual gain mapping is determined by comparing the cell region of the reconstructed gain mapping with the corresponding cell region of the gain mapping.

17. The non-transitory computer-readable storage medium of claim 16, wherein the second image can be reproduced using a combination of the first image, the image-specific global tone curve, and the residual gain map.

18. The non-transitory computer-readable storage medium of claim 16, wherein the residual gain mapping includes the difference between the gain value of the cell region of the reconstructed gain mapping and the gain value of the corresponding cell region of the gain mapping.

19. The non-transitory computer-readable storage medium according to claim 15, wherein: The first image includes a standard dynamic range (SDR) image; and The second image includes a high dynamic range (HDR) image.

20. The non-transitory computer-readable storage medium of claim 15, wherein the step further comprises: A variant second image is generated based on the first image and the image-specific global tone curve.

21. The non-transitory computer-readable storage medium of claim 20, wherein the variant second image differs from the second image in one or more channel values ​​in the corresponding cell region.

22. The non-transitory computer-readable storage medium of claim 15, wherein the image-specific global tone curve comprises a plurality of consecutive segments, each segment corresponding to a cubic spline.

23. The non-transitory computer-readable storage medium of claim 15, wherein the curve fitting constrains the image-specific global tone curve to a monotonically increasing function.

24. The non-transitory computer-readable storage medium of claim 15, wherein the cost function comprises one of the following: a least squares function, a structural similarity index measure (SSIM), or a variance inflation factor (VIF).

25. The non-transitory computer-readable storage medium of claim 15, wherein the channel values ​​of the first image for generating the scatter plot include the brightness values ​​of the cell regions of the first image.

26. The non-transitory computer-readable storage medium of claim 15, wherein the channel values ​​of the first image for generating the scatter plot include one or more color values ​​of a cell region of the first image.

27. The non-transitory computer-readable storage medium of claim 15, wherein each cell region corresponds to a single pixel.

28. The non-transitory computer-readable storage medium of claim 15, wherein each cell region corresponds to a plurality of consecutive pixels.

29. A computing device configured to generate a specific global tone curve for an image, the computing device comprising: At least one processor; and At least one memory storing instructions that, when executed by the at least one processor, cause the computing device to perform steps, the steps including: Gain mapping is obtained based on a comparison between a unit region of a first image and a corresponding unit region of a second image. The gain mapping includes a set of gain values ​​for the unit regions of the first image and the second image. Generate a scatter plot comprising a set of points, each point including a gain value from the gain map and a brightness value for the corresponding cell region of the first image; The specific global tone curve of the image is determined by curve fitting the scatter plot using a cost function; and The image-specific global tone curve is stored together with the first image.

30. The computing device of claim 29, wherein the step further comprises: Generate a reconstructed gain map from the specific global tone curve of the image; as well as The residual gain mapping is determined by comparing the cell region of the reconstructed gain mapping with the corresponding cell region of the gain mapping.

31. The computing device of claim 30, wherein the second image can be reproduced using a combination of the first image, the image-specific global tone curve, and the residual gain mapping.

32. The computing device of claim 30, wherein the residual gain mapping includes the difference between the gain value of the cell region of the reconstructed gain mapping and the gain value of the corresponding cell region of the gain mapping.

33. The computing device according to claim 29, wherein: The first image includes a standard dynamic range (SDR) image; and The second image includes a high dynamic range (HDR) image.

34. The computing device of claim 29, wherein the step further comprises: A variant second image is generated based on the first image and the image-specific global tone curve.

35. The computing device of claim 34, wherein the variant second image differs from the second image in terms of one or more channel values ​​in the corresponding cell region.

36. The computing device of claim 29, wherein the image-specific global tone curve comprises a plurality of consecutive segments, each segment corresponding to a cubic spline.

37. The computing device of claim 29, wherein the curve fitting constrains the image-specific global tone curve to a monotonically increasing function.

38. The computing device of claim 29, wherein the cost function comprises one of the following: a least squares function, a structural similarity index measure (SSIM), or a variance inflation factor (VIF).

39. The computing device of claim 29, wherein the channel values ​​of the first image for generating the scatter plot include the brightness values ​​of the cell regions of the first image.

40. The computing device of claim 29, wherein the channel values ​​of the first image for generating the scatter plot include one or more color values ​​of a cell region of the first image.

41. The computing device of claim 29, wherein each unit region corresponds to a single pixel.

42. The computing device of claim 29, wherein each unit region corresponds to a plurality of consecutive pixels.

43. A computing device configured to generate a specific global tone curve for an image, the computing device comprising: A component for obtaining a gain mapping based on a comparison between a cell region of a first image and a corresponding cell region of a second image, the gain mapping including a set of gain values ​​for the cell regions of the first image and the second image; A component for generating a scatter plot comprising a set of points, each point including a gain value from the gain map and a brightness value for the corresponding cell region of the first image; A component for determining a specific global tone curve of the image based on curve fitting of the scatter plot using a cost function; and A component for storing the image-specific global tone curve together with the first image.

44. The computing device of claim 43, further comprising components for performing the following operations: Generate a reconstructed gain map from the image-specific global tone curve; and The residual gain mapping is determined by comparing the cell region of the reconstructed gain mapping with the corresponding cell region of the gain mapping.

45. The computing device of claim 44, wherein the second image can be reproduced using a combination of the first image, the image-specific global tone curve, and the residual gain map.

46. ​​The computing device of claim 44, wherein the residual gain mapping includes the difference between the gain value of the cell region of the reconstructed gain mapping and the gain value of the corresponding cell region of the gain mapping.

47. The computing device of claim 43, wherein: The first image includes a standard dynamic range (SDR) image; and The second image includes a high dynamic range (HDR) image.

48. The computing device of claim 43, further comprising components for performing the following operations: A variant second image is generated based on the first image and the image-specific global tone curve.

49. The computing device of claim 48, wherein the variant second image differs from the second image in terms of one or more channel values ​​in the corresponding cell region.

50. The computing device of claim 43, wherein the image-specific global tone curve comprises a plurality of consecutive segments, each segment corresponding to a cubic spline.

51. The computing device of claim 43, wherein the curve fitting constrains the image-specific global tone curve to a monotonically increasing function.

52. The computing device of claim 43, wherein the cost function comprises one of the following: a least squares function, a structural similarity index measure (SSIM), or a variance inflation factor (VIF).

53. The computing device of claim 43, wherein the channel values ​​of the first image for generating the scatter plot include the brightness values ​​of the cell regions of the first image.

54. The computing device of claim 43, wherein the channel values ​​of the first image for generating the scatter plot include one or more color values ​​of a cell region of the first image.

55. The computing device of claim 43, wherein each unit region corresponds to a single pixel.

56. The computing device of claim 43, wherein each unit region corresponds to a plurality of consecutive pixels.

57. A method for generating a specific global tone curve for an image, the method comprising, at a computing device: Gain mapping is obtained based on a comparison between pixels in a first image and corresponding pixels in a second image, wherein the gain mapping includes a set of gain values ​​for the pixels in the first image and the second image; Identify multiple regions of the first image; Generate for each region of the first image: The corresponding scatter plot, wherein each corresponding scatter plot includes a set of points, each point including a gain value and a brightness value for the corresponding region of the first image, and A corresponding image-specific global tone curve for the corresponding scatter plot using the corresponding cost function; and The image-specific global tone curve is stored together with the first image.

58. The method of claim 57, further comprising, before generating the scatter plot and the image-specific global tone curve: Identify the corresponding weights to be applied to one or more pixels in the first image, the second image, and / or the gain mapping; and The corresponding weights are applied to the one or more pixels.

59. The method of claim 57, further comprising: Receive the selection of the image-specific global tone curve from the image-specific global tone curve; as well as Apply the image-specific global tone curve to the first image.

60. A non-transitory computer-readable storage medium configured to store instructions, which, when executed by at least one processor included in a computing device, cause the computing device to perform steps to generate an image-specific global tone curve, the steps comprising: Gain mapping is obtained based on a comparison between pixels in a first image and corresponding pixels in a second image, wherein the gain mapping includes a set of gain values ​​for the pixels in the first image and the second image; Identify multiple regions of the first image; Generate for each region of the first image: The corresponding scatter plot, wherein each corresponding scatter plot includes a set of points, each point including a gain value and a brightness value for the corresponding region of the first image, and A corresponding image-specific global tone curve for the corresponding scatter plot using the corresponding cost function; and The image-specific global tone curve is stored together with the first image.

61. The non-transitory computer-readable storage medium of claim 60, wherein the step further comprises generating the scatter plot and the image-specific global tone curve: Identify the corresponding weights to be applied to one or more pixels in the first image, the second image, and / or the gain mapping; and The corresponding weights are applied to the one or more pixels.

62. The non-transitory computer-readable storage medium of claim 60, wherein the step further comprises: Receive the selection of the image-specific global tone curve from the image-specific global tone curve; as well as Apply the image-specific global tone curve to the first image.

63. A computing device configured to generate a specific global tone curve for an image, the computing device comprising: At least one processor; and At least one memory storing instructions that, when executed by the at least one processor, cause the computing device to perform steps, the steps including: Gain mapping is obtained based on a comparison between pixels in a first image and corresponding pixels in a second image, wherein the gain mapping includes a set of gain values ​​for the pixels in the first image and the second image; Identify multiple regions of the first image; Generate for each region of the first image: The corresponding scatter plot, wherein each corresponding scatter plot includes a set of points, each point including a gain value and a brightness value for the corresponding region of the first image, and A corresponding image-specific global tone curve for the corresponding scatter plot using the corresponding cost function; and The image-specific global tone curve is stored together with the first image.

64. The computing device of claim 63, wherein the step further comprises generating the scatter plot and the image-specific global tone curve: Identify the corresponding weights to be applied to one or more pixels in the first image, the second image, and / or the gain mapping; and The corresponding weights are applied to the one or more pixels.

65. The computing device of claim 63, wherein the step further comprises: Receive the selection of the image-specific global tone curve from the image-specific global tone curve; as well as Apply the image-specific global tone curve to the first image.

66. A computing device configured to generate a specific global tone curve for an image, the computing device comprising: A component for obtaining a gain mapping based on a comparison between pixels in a first image and corresponding pixels in a second image, the gain mapping comprising a set of gain values ​​for the pixels in the first image and the second image; Components used to identify multiple regions of the first image; The following components are generated for each region of the first image: The corresponding scatter plot, wherein each corresponding scatter plot includes a set of points, each point including a gain value and a brightness value for the corresponding region of the first image, and The corresponding image-specific global tone curve for the corresponding scatter plot using the corresponding cost function; and A component for storing the image-specific global tone curve together with the first image.

67. The computing device of claim 66, further comprising components for performing the following operations prior to generating the scatter plot and the image-specific global tone curve: Identify the corresponding weights to be applied to one or more pixels in the first image, the second image, and / or the gain mapping; and The corresponding weights are applied to the one or more pixels.

68. The computing device of claim 66, further comprising components for performing the following operations: Receive the selection of the image-specific global tone curve from the image-specific global tone curve; and Apply the image-specific global tone curve to the first image.

Images