Method, computer program, and electronic device for tone mapping

Abstract

A method, computer program, and electronic device are proposed for tone mapping an HDR input image to an LDR output image, the method comprising: a) obtaining a luminance component of the input image; b) obtaining an initial tone mapping curve, which is a global tone mapping curve; c) obtaining a luminance histogram representing the luminance distribution of the input image; d) determining a plurality of clusters of the luminance distribution from the luminance histogram, where each cluster has a centroid; e) generating an adapted tone mapping curve for each cluster by adapting a slope of the initial tone mapping curve depending on a concentration of luminance values ​​in that cluster, where the higher the concentration of luminance values, the higher the slope of the adapted tone mapping curve; and f) generating an output image by applying the adapted tone mapping curve to at least the luminance component of the input image.

Description

[Technical Field]

[0001] The present invention relates to a method for tone mapping a high dynamic range (HDR) input image to a low dynamic range (LDR) output image.

[0002] Furthermore, the invention relates to a computer program having program code means adapted to carry out such a method.

[0003] Furthermore, the invention relates to an electronic device adapted to carry out such a method. [Background technology]

[0004] Generally, the present invention relates to the field of high dynamic range (HDR) imaging. HDR images typically offer a dynamic range of 14-16 bits to 20-24 bits, whereas conventional LDR images (or standard dynamic range images) typically offer only an 8-10 bit dynamic range. HDR images therefore allow for much better preservation of details in real-world scenes that contain very bright areas, but also very dark areas, than conventional LDR images. Luminance is a photometric quantity used in photography and video technology to measure the brightness of a pixel. HDR images allow for the capture of a dynamic range similar to that of the human eye, which is approximately 10 14 It can detect a brightness range up to

[0005] However, typical displays can only reproduce the luminance range of conventional LDR images. Currently, there are no available displays that can reproduce the luminance range of HDR images, and only a very few expensive displays can reproduce a luminance range of about 12 bits.

[0006] Due to these display contrast limitations, the luminance range of the input HDR image must be reduced in order to be displayed on a regular display. This process of converting an HDR image to an LDR image is commonly referred to as tone mapping.

[0007] In the context of this application, which relates to a method for tone mapping a high dynamic range input image to a low dynamic range output image, the term "low dynamic range output image" can generally refer to any image having a dynamic range that is smaller than the dynamic range of the high dynamic range input image. In particular, in certain embodiments, the low dynamic range can be a conventional low dynamic range image and / or a standard dynamic range (SDR) image, e.g., having a dynamic range of 8-10 bits.

[0008] The input and output images may be part of an input and output video sequence, respectively.

[0009] The objectives of a tone mapping method may differ depending on the particular application: for example, different criteria must be met to generate images compatible with the Human Visual System (HVS) on the one hand, and for Machine Vision System (MVS) applications on the other.

[0010] With respect to HVS, to display an HDR image on a standard display with a displayable dynamic range of, for example, 8-10 bits, the HDR image, which may have a luminance range of 20-24 bits, must be compressed so that the dark, mid-tone, and bright image content are all clearly distinguishable by the naked human eye. At the same time, the reproduction of the image on the display should appear as realistic and natural as possible to a human observer.

[0011] On the other hand, for MVS, the tone mapping method that compresses an HDR image (e.g., with a 20-24 bit luminance range) to an LDR image (e.g., with an 8-10 bit luminance range) should allow MVS to process the tone-mapped image as effectively as possible. In most cases, this means that all structures in the image (edges, textures) must be preserved as well as possible.

[0012] In general, a common goal of tone mapping in both HVS and MVS applications is to minimize the loss of image information. To this end, it is essential to maintain contrast, especially local contrast, adequate for the intended use.

[0013] The state of the art is known to have developed a variety of different tone mapping operators to meet these challenges, which can be classified into two main types: global tone mapping operators and local tone mapping operators.

[0014] On the other hand, a global tone mapping operator (or global tone mapping method) maps all pixels of an image in the same way, regardless of their position in the image or surrounding pixels. Generally, a global tone mapping operator is a nonlinear function (also called a tone mapping curve) that is based on the global luminance and / or other global characteristics of a particular image. Because the same function is applied to all pixels of a particular image, global tone mapping methods are simple, fast, and require only a small amount of processing power. Therefore, they can be implemented using low-cost hardware, for example, based on lookup tables. However, because only the global characteristics of the image are considered, these global methods often cause a significant loss of contrast, especially with respect to local image details.

[0015] On the other hand, local tone mapping operators (or local tone mapping methods) map each pixel of an image differently, depending specifically on local image characteristics, e.g., on surrounding pixels. When designed correctly, these methods can achieve very good results in terms of preserving local contrast, which is beneficial for the intended application. However, these local tone mapping methods are prone to artifacts such as halo effects and ringing, and their output can appear unrealistic to a human observer. Furthermore, these methods are generally more complex than global tone mapping methods and therefore often require significantly more processing power.

[0016] For example, U.S. Patent Application Publication No. 2018 / 0097992 discloses a global tone mapping method and corresponding system that includes calculating a first histogram of luminance values ​​of an input image, accessing a target histogram of the image, and calculating a transfer function based on the first histogram and the target histogram. The tone-mapped image is generated by applying the transfer function to pixel values ​​of the input image.

[0017] It is an object of the present invention to provide a method for tone mapping an HDR input image to an LDR output image that provides improved preservation of image detail and contrast compared to conventional global tone mapping methods, but requires a smaller amount of processing power than complex local tone mapping methods.

[0018] The object of the invention is achieved by a method for tone mapping a high dynamic range (HDR) input image to a low dynamic range (LDR) output image having the features of claim 1. Summary of the Invention

[0019] According to the present invention, the method comprises at least the following steps: a) obtaining the luminance component of an input image; b) obtaining an initial tone mapping curve, which is a global tone mapping curve; c) obtaining a luminance histogram representing the luminance distribution of the input image; d) determining a plurality of clusters of luminance distributions from the luminance histogram, wherein each cluster has a centroid; e) generating an adapted tone mapping curve for each cluster by adapting a slope of the initial tone mapping curve according to the concentration of luminance values ​​in that cluster, wherein the higher the concentration of luminance values ​​in a cluster, the higher the slope of the adapted tone mapping curve for that cluster; f) generating an output image by applying the adapted tone mapping curve to at least the luminance component of the input image; The steps of the method need not be performed in any particular order, and the invention is not limited accordingly, i.e., the alphabetical order of the letters does not imply any particular sequence of steps a) through f). For example, it should be understood that step b) may be performed after steps c) and d), or some of the method steps may be performed in parallel.

[0020] In step a), a luminance component of an input image is obtained. Such a luminance component can be, for example, a luma component (luma channel) of a color space in which luminance intensity values ​​and tonal values ​​are separated, such as the luma component Y of the YCbCr color space (or Y' of the Y'CbCr color space). If the luminance component is a luma component of a color space in which luminance intensity values ​​and tonal values ​​are separated, such as the luma component Y of the YCbCr color space, the input HDR image needs to be first converted to such a color space. For example, if the input image is an RGB image, the RGB input image can be first converted to the YCbCr color space.

[0021] However, the luminance component can generally be a combination of components of any color space or other component that represents the luminance of an image. For example, the luminance component can be the green component G in the RGB color space, since this can represent the luminance of an image sufficiently well. Naturally, the luminance component comprises multiple luminance values. In particular, the luminance component of an input image can comprise one luminance value for each pixel of the input image.

[0022] The terms "dynamic range" and "luminance range" are used equivalently in the context of this application.

[0023] In step b), an initial tone mapping curve is obtained, which is a global tone mapping curve. In other words, in step b), a global tone mapping function (or global tone mapping operator) that maps input luminance values ​​of the HDR input image to output luminance values ​​of the LDR output image is obtained.

[0024] In an advantageous embodiment of the present invention, the initial tone mapping curve and / or the adapted tone mapping curve, respectively, may be implemented as a look-up table. Such an embodiment of the present invention provides the advantage that the tone mapping method can be implemented in a computationally efficient manner.

[0025] Furthermore, in another advantageous embodiment of the present invention, the initial tone mapping curve and / or the adapted tone mapping curve can be constructed by subsampling the HDR input image and interpolating between the subsampled values. Since the dynamic range of the HDR input image can be very large, such an embodiment of the present invention provides the advantage that the required processing power can be further reduced.

[0026] In step c), a luminance histogram representing the luminance distribution of the input image is obtained. Typically, this luminance histogram can be obtained from the original HDR input image. However, as will be explained in more detail below, the luminance histogram can also be obtained from a compressed version of the input image in order to reduce the processing power required for step c).

[0027] In step d), a plurality of clusters of the luminance distribution are determined from the luminance histogram. The number of clusters created in step d) may be a predetermined number. In particular, the number of clusters may be greater than 2 and less than 12. In particular, the number of clusters may be greater than 2 and less than 6. In particular, the number of clusters may be greater than 2 and less than 5. However, the number of clusters may not be predetermined. For example, the number of clusters may be determined by the clustering algorithm applied.

[0028] In step e), an adapted tone mapping curve is generated from the initial tone mapping curve. To this end, for each of the plurality of clusters determined in step d), the slope of the initial tone mapping curve is adapted depending on the concentration of luminance values ​​in that cluster: the higher the concentration of luminance values ​​in the respective cluster, the greater the slope of the adapted tone mapping curve for that cluster. The concentration of luminance values ​​in each cluster can be determined based on different metrics, including the amount of variation of luminance values ​​in the cluster and / or the absolute or relative number of luminance values ​​in that cluster, as will be described in more detail below.

[0029] In step f), an output image is generated by applying the adapted tone mapping curve to at least the luminance component of the input image.

[0030] The method according to the present invention effectively avoids the loss of relevant image information, such as object contours and textures. For this purpose, it is essential to preserve the local contrast, which also comprises the texture structure of the image. The present invention is based on the insight that, in order to avoid the loss of local contrast and therefore the loss of local image information as much as possible, it is important that the tone mapping curve is sufficiently steep, i.e., has a sufficiently large gradient, for those portions of the luminance range (HDR) that are dominant in each HDR input image. These portions of the luminance range are also referred to as dominant tonal ranges. The inventors have found that these dominant portions of the luminance range can be determined based on the density of luminance values ​​from a luminance histogram. There are two reasons for this. On the one hand, if the luminance histogram shows high density luminance values ​​in certain portions of the luminance range, this means that compressing the dynamic range in these portions will affect particularly large regions of the input image. On the other hand, if the luminance histogram shows locally high density luminance values, this means that in these portions of the luminance range, the input image contains areas of low local contrast, where pixels and their surrounding luminance values ​​are particularly close. These regions of the input image are particularly sensitive to compression.

[0031] To identify different portions of the luminance range that are dominant in the input image, a clustering method is applied to determine multiple clusters of luminance distribution from the luminance histogram. For this purpose, known clustering algorithms, such as k-means clustering, can be used. A high concentration of luminance values ​​in a cluster indicates that the corresponding portion of the luminance range is dominant in the input image. By increasing the slope of the tone mapping curve for those portions of the luminance range with high concentrations of luminance values ​​in the input image, the dominant tonal range of the input image is compressed to a lesser extent, thereby preserving contrast. In other words, those portions of the luminance range that contain particularly large amounts of information in the input image are preserved without being overly compressed.

[0032] As a result, the present invention effectively avoids the loss of relevant image information, such as object contours, textures and details, by preserving the contrast in the dominant tonal range of the image, which is determined based on the density of the luminance values ​​of each cluster of the luminance distribution.

[0033] This effect is achieved by global tone mapping because the adapted tone mapping curve, like the initial tone mapping curve, is a global tone mapping curve. Compared to local tone mapping methods, this results in significantly lower processing power requirements. Therefore, the present invention can be advantageously implemented using low-cost hardware.

[0034] According to an advantageous embodiment of the invention, it is proposed that step c) comprises the following steps: compressing the input image; Obtaining a luminance histogram from a compressed input image. Such an embodiment of the invention provides the advantage that the processing power required to process the histogram and determine clusters of brightness values ​​in step d) can be significantly reduced.

[0035] In general, the luminance histogram can also be obtained directly from the HDR input image. However, because the size of the HDR tonal range is very large (it may use 20 bits, 24 bits, or even more bits), a large amount of processing power is required to process the histogram, especially to determine the clusters of luminance values ​​from the histogram in step d). Such processing power may not be available in typical image processing pipelines, and it may be undesirable or even impossible to equip an image pipeline with such processing power.

[0036] Nevertheless, the scope of this application also includes obtaining a luminance histogram from an uncompressed input image, and therefore also includes obtaining a luminance histogram directly from an HDR input image.

[0037] According to another advantageous embodiment of the invention, it is proposed that the input image is compressed by applying a global tone mapping curve. In particular, the input image can be compressed by applying an initial tone mapping curve. In this case, the same tone mapping curve is used to compress the input image which is also used as the basis for generating the adapted tone mapping curve in step e).

[0038] Such an embodiment of the present invention provides the advantage that the input image can be compressed in a particularly efficient and computationally cheap way, since the global tone mapping curve can be implemented, for example, as a look-up table, thereby reducing the required processing power.

[0039] According to another advantageous embodiment of the invention, it is proposed that the initial tone mapping curve is based on the global tone mapping operator according to Reinhard.

[0040] The global tone mapping operator according to Reinhard (also called the Reinhard tone mapping operator) is given by the following equation:

[0041]

number

[0042] where L out is the displayable output luminance for pixel (x,y), and L in (x,y) is the (scaled) input luminance, and L whiteis the minimum luminance that maps to pure white. This global tone mapping operator is presented in the paper "Photographic Tone Reproduction for Digital Images" by E. Reinhard, J. Ferwerda, and P. Shirley in ACM Transactions on Graphics, May 2002. The term "Reinhard curve" refers to the global tone mapping curve constructed by applying Reinhard's global tone mapping operator.

[0043] According to another advantageous embodiment of the invention, it is proposed that the initial tone mapping curve is a Reinhard curve.

[0044] The Reinhard global tone mapping operator is relatively simple yet provides good results in terms of output image quality relative to conventional global tone mapping operators. Thus, such embodiments of the present invention that use the Reinhard global tone mapping operator for the initial tone mapping operator that forms the basis of the adapted tone mapping curve used to generate the output image offer the advantage that they are simple to implement while at the same time resulting in particularly good quality of the output image, particularly with regard to preserving image contrast and associated detail.

[0045] According to another advantageous embodiment of the invention, it is proposed that in step d) the clusters are determined by k-means clustering.

[0046] Such an embodiment of the present invention provides the advantage that a computationally efficient implementation of the well-known k-means clustering algorithm can be used to effectively determine the number of clusters in the luminance distribution of the input image from the luminance histogram.

[0047] According to another advantageous embodiment of the invention, step d) comprises merging two or more clusters depending on the distance between these clusters and / or the centroids of these clusters, in particular if the distance is smaller than a threshold value.

[0048] In such an embodiment, merging two or more clusters results in reducing the number of clusters if the clusters are too close to each other. Therefore, such an embodiment of the present invention provides the advantage that the number of clusters can be adapted to each input image and its brightness distribution. As a result, an inappropriate large number of clusters can be avoided.

[0049] According to another advantageous embodiment of the invention, the concentration of luminance values ​​in a cluster is determined based on at least the following: - the amount of variance in the brightness values ​​in that cluster, and / or Specifically, the number of luminance values ​​in that cluster relative to the total number of luminance values ​​in the luminance histogram.

[0050] As mentioned above, the present invention is based on the discovery that in order to preserve important image details and local contrast, a sufficiently large slope of the tone mapping curve must be ensured for those parts of the input image luminance histogram that exhibit high density luminance values.

[0051] For this purpose, the concentration of the luminance values ​​can be determined, for example, based on the amount of variation of the luminance values ​​in each cluster. The concentration can also be determined based on the dispersion (also referred to as variability, scattering, or spread) of the luminance values ​​in that cluster, since this reflects the amount of variation. The amount of variation and / or dispersion of the luminance values ​​can be quantified, for example, using common measures of statistical dispersion. Examples of such measures include standard deviation, variance, and coefficient of variation.

[0052] The variation and / or variance of the brightness values ​​can be estimated, for example, by empirically determining typical variations from multiple images. In particular, different variations can be estimated for different cluster centroids, i.e., for different cluster positions within the brightness range. Alternatively or additionally, the variation and / or variance can be determined analytically from the input image (or, as described above, from a compressed version of the input image).

[0053] Alternatively or additionally, the density of the luminance values ​​can be determined based on the number of luminance values ​​in each cluster. In particular, the density of the luminance values ​​can be determined based on the number of luminance values ​​in that cluster relative to the total number of luminance values ​​in the luminance histogram. In other words, the density of the luminance values ​​can be determined based on the ratio of the luminance values ​​in each cluster.

[0054] Such an embodiment of the invention provides the advantage that the density of the luminance values ​​can be reliably and efficiently determined and therefore the slope of the adapted tone mapping curve can be appropriately set in step e).

[0055] According to another advantageous embodiment of the invention, in step e), it is proposed that for each cluster, the gradient of the initial tone mapping curve is adapted within an adaptation area depending on the concentration of luminance values ​​in the adaptation area, wherein the adaptation area comprises a range of luminance values ​​around the center of gravity of the cluster.

[0056] In other words, it is proposed that the density of the luminance values ​​is determined and the gradient of the tone mapping curve is adapted to the area around the centroid of the cluster.

[0057] According to another advantageous embodiment of the invention, it is proposed that the size and / or position of the adaptation area is limited by lower and upper boundaries representing brightness values, wherein the lower and / or upper boundaries are determined depending on: - the centroid of each cluster (i.e., the position of the cluster within the luminance range), and / or - the estimated or actual variation of said luminance values ​​in each cluster; and / or - the distance between the centroids of different clusters, in particular the distance between the centroid of each cluster and the centroid of the adjacent cluster.

[0058] The above-described embodiments of the present invention that rely on determining the density of luminance values ​​and adapting the slope of the tone mapping curve in an adaptation region around the cluster centroid provide the advantage that the tone mapping curve is specifically adapted to those parts of the luminance range of the input image that represent the dominant tonal range that is expected to be located around the cluster centroid, so that particularly effective preservation of image contrast can be achieved.

[0059] According to another advantageous embodiment of the invention, it is proposed that step e) comprises determining a plurality of luminance zones, each luminance zone covering a part of the luminance range, and adapting the slope of the initial tone mapping curve by adapting its output value for each luminance zone.

[0060] Such an embodiment of the invention simplifies the processing of the luminance range of the input image, which can be very large, and further offers the advantage of allowing the slope of the tone mapping curve to be adapted in a flexible way, since the luminance zones can be defined depending on the individual characteristics of the input image.

[0061] According to another advantageous embodiment of the invention, it is proposed that step a) comprises obtaining a luminance component and a plurality of chrominance components of the input image, and that step f) comprises applying an adapted tone mapping curve to the luminance component and the plurality of chrominance components.

[0062] In other words, it is proposed that the method according to the invention is applied not only to the luminance component of the input image, but also to at least one chrominance component, In particular, the plurality of chrominance components may comprise all chrominance components of the input image.

[0063] Such an embodiment of the present invention provides the advantages mentioned above, in particular that preserving important image details and local contrast can be used for tone mapping of the color information of the input image. This allows for very efficient processing of all components of the input image, since the same adapted tone mapping curve can be used for the chrominance components as for the luminance component.

[0064] According to another advantageous embodiment of the invention, it is proposed that step f) further comprises desaturating the resulting chrominance components of the output image, in particular by applying a desaturation scaling factor to the chrominance components.

[0065] In general, tone mapping a colored HDR input image to an LDR output image can have the undesirable effect of making certain regions of the output image appear oversaturated, which may be particularly true for bright regions of the input image. To compensate for this, the chrominance components of the output image may be desaturated, for example, by applying a desaturation scaling factor that may be determined experimentally. For example, the scaling factor may be implemented by a lookup table.

[0066] Such an embodiment of the present invention therefore provides the advantage of being able to compensate for undesired oversaturation of the chrominance components of the output image.

[0067] According to another advantageous embodiment, it is proposed that step e) comprises, after adapting the gradient of the initial tone mapping curve, smoothing the resulting adapted tone mapping curve, in particular by applying Berstein and / or Berstein-Bézier polynomials and / or Bézier curves.

[0068] Such an embodiment of the present invention provides the advantage that a smooth adapted tone mapping curve can be generated and applied to the output image.

[0069] The object of the invention is further achieved by a computer program having program code means adapted to carry out the above-mentioned method when the computer program is run on a computer.

[0070] The object of the invention is further achieved by an electronic device adapted to carry out the method described above.

[0071] The electronic device can be, for example, a stand-alone integrated circuit (IC) or part thereof. The electronic device can also be a system on a chip (SoC) or part thereof. The electronic device can also be part of an image processing pipeline and / or an image processing chain. The electronic device can also be a camera or a display or part of a camera or a display. The electronic device can also be a system comprising a camera and / or a display. The electronic device can also be part of such a system. [Brief explanation of the drawings]

[0072] The invention will now be explained in more detail by means of exemplary embodiments which are illustrated diagrammatically in the accompanying drawings, which show: [Figure 1] FIG. 1 is a schematic diagram of a method for tone mapping according to the present invention. [Figure 2] FIG. 2 is a schematic diagram of a luminance histogram. [Figure 3] FIG. 3 is a schematic diagram of a luminance histogram and multiple clusters with corresponding centroids and adaptive regions. [Figure 4] FIG. 4 is a schematic diagram of the initial tone mapping curve and the adapted tone mapping curve. [Figure 5] FIG. 5 is a schematic diagram of an exemplary image processing system including an electronic device according to the present invention. Detailed Description of the Embodiments

[0073] 1 shows a schematic diagram of an exemplary method for tone mapping an HDR input image to an LDR output image according to this invention. In this exemplary embodiment, the HDR input image has a 16-bit dynamic range, while the LDR output image has only an 8-bit dynamic range.

[0074] In steps 101 and 102, which correspond to step a) described above, a luminance component and multiple chrominance components of an input image, two chrominance components in this exemplary embodiment, are obtained. To this end, in step 101, an HDR input image having an exemplary dynamic range of 16 bits is read, which initially has an RGB format. In the exemplary embodiment shown in FIG. 1 , the luminance channel Y (also referred to as the luma component Y) of the YCbCr color model is used as the luminance component of the input image, and the Cb and Cr channels (also referred to as the Cb and Cr components) are used as two chrominance components of the input image. Therefore, in step 102, the RGB input image is converted to the YCbCr color model, which provides an HDR input image in YCbCr format having the Y channel as the luminance component and the Cb and Cr channels as chrominance components.

[0075] In step 103 shown in Figure 1, which corresponds to step b) described above, an initial tone mapping curve is obtained, which is a global tone mapping curve. In this exemplary embodiment, the global tone mapping operator due to Reinhard is used for this purpose, which is given by:

[0076]

number

[0077] where L out is the displayable output luminance for pixel (x,y), and L in (x,y) is the (scaled) input luminance, and L white is the minimum luminance that maps to pure white.

[0078] The next steps 104 and 105 shown in Figure 1 correspond to step c) described above. In this exemplary embodiment, in step 104, the input image is compressed by applying an initial tone mapping curve, i.e., by applying the Reinhard global tone mapping operator to the luminance component Y of the HDR input image.

[0079] Then, in step 105, a luminance histogram representing the luminance distribution of the input image is obtained from the compressed input image.

[0080] Figure 2 shows a schematic diagram of an exemplary luminance histogram 3 as obtained in step 105 shown in Figure 1, i.e., step c) of the method according to the invention described above. In the exemplary embodiment shown in Figure 2, the luminance histogram 3 was obtained from a compressed input image with an 8-bit luminance range. The horizontal axis 51 of the diagram in Figure 2 shows the luminance values ​​Y between 0 and 255, where 0 represents the darkest pixel and 255 represents the brightest pixel. The vertical axis 52 shows the number of pixels for each luminance value.

[0081] 1, in step 106, a plurality of clusters of luminance distributions are determined from the luminance histogram 3. Each of the determined clusters has a centroid. In the exemplary embodiment described herein, the plurality of clusters are determined using a k-means clustering algorithm.

[0082] In this exemplary embodiment, the k-means clustering algorithm initially starts with a predetermined number of four clusters. However, an additional constraint is defined to prevent determining clusters that are too close to each other. To this end, at each iteration of the algorithm, it is checked whether the distance between any pair of cluster centroids is less than a predetermined threshold. In this case, the two clusters (or their centroids) are assumed to be too close to each other, and the two clusters are merged. To this end, the new centroids resulting from the merge are calculated as follows:

[0083]

number

[0084] where c ij is the centroid of the merged cluster, and c i and c j are the centroids of clusters i and j that are merged because they are too close to each other, respectively, and n i and n j are the numbers of intensity values ​​(pixels) in clusters i and j, respectively.

[0085] As a result, in the exemplary embodiment described herein, the number of clusters determined from the luminance histogram is reduced from an initial number of four clusters to a final number of three clusters, as otherwise two of the cluster centroids would be too close to each other. Thus, after completing step 106, three clusters of the luminance distribution of the input image defined by the cluster centroids have been determined.

[0086] In step 107 of the exemplary embodiment shown in Figure 1, an adaptation region is determined for each of the three clusters of the luminance distribution of the input image. Each of the three adaptation regions is bounded by a lower boundary and an upper boundary representing luminance values.

[0087] In this exemplary embodiment, the lower and upper boundaries are determined according to the centroids of the respective clusters (their luminance values) and the estimated amount of variation of the luminance values ​​within the respective clusters. To this end, the following closeness estimation metric is defined for each luminance value Y and each cluster i:

[0088]

number

[0089] where Y is the brightness value from the brightness histogram of the input image, and c i is the centroid of cluster i, n is the number of clusters,

[0090]

number

[0091] is the estimated variation of luminance values ​​in cluster i, which serves as a metric for the amount of estimated variation. In this exemplary embodiment, Y is the luminance value of the compressed input image, as described above. However, in alternative embodiments, the original uncompressed HDR input image may also be used for this purpose. The approximate estimation metric f close Those luminance values ​​for which σ is greater than a predetermined threshold are included in the adaptive region. In particular, this threshold can be defined according to the number of clusters. For example, the threshold can be defined as 0.5 if the number of clusters is greater than 3, as 0.1 if the number of clusters is 3, and as 0.001 if the number of clusters is less than 3.

[0092] In an alternative embodiment, the lower and upper boundaries can be determined according to the centroid of each cluster (the brightness value of the centroid), the estimated or actual amount of variation in brightness values ​​in each cluster, and the distance between the centroid of each cluster and the centroid of an adjacent cluster. For example, the upper and lower boundaries of the adaptive region of a cluster can also be determined based on the following formula:

[0093]

number

[0094] where b u,i and b l,i are the upper and lower boundaries of cluster i, respectively, and c i is the cluster centroid, and σ i is the estimated standard deviation of the luminance values ​​in cluster i, and d i is the distance between the centroid of cluster i and the centroid of the adjacent cluster.

[0095] The results of steps 101-107 are illustrated diagrammatically in Figure 3. In addition to the luminance histogram 3 of Figure 2, Figure 3 shows the three clusters of luminance distributions determined in step 106 and their centroids c1, c2, and c3, where c1 is the centroid of the first cluster, c2 is the centroid of the second cluster, and c3 is the centroid of the third cluster. Furthermore, Figure 3 diagrammatically shows three adaptive regions aa1, aa2, and aa3 determined as described above. Each of the adaptive regions is bounded by a lower boundary and an upper boundary, where the adaptive region aa1 of the first cluster is bounded by a lower boundary b. l1 and upper boundary b u1 The adaptive region aa2 of the second cluster is bounded by the lower boundary b l2 and upper boundary b u2 The adaptive region aa3 of the third cluster is bounded by the lower boundary b l3 and upper boundary b u3 is limited by

[0096] 1, in step 108, the density of luminance values ​​is determined for each adaptive region. In the exemplary embodiment described herein, the density of luminance values ​​is determined from the luminance histogram 3. In particular, for each of the three adaptive regions aa1, aa2, aa3, the density of luminance values ​​in that adaptive region, which corresponds to the density of luminance values ​​in the corresponding cluster in this exemplary embodiment, is determined based on the number of luminance values ​​in the adaptive region relative to the total number of luminance values ​​in the luminance histogram 3. In this exemplary embodiment, the number of luminance values ​​in the adaptive region therefore corresponds to the number of luminance values ​​in the corresponding cluster.

[0097] In other embodiments, the concentration of the luminance values ​​may additionally or alternatively be determined based on the amount of variation in the luminance values ​​in the cluster, in particular based on the amount of variation in the adaptive region of the cluster, as already described above.

[0098] Then, in step 109 shown in FIG. 1, for each cluster, an adapted tone mapping curve is generated by adapting the slope of the initial tone mapping curve according to the density of the luminance values ​​in that cluster, where the higher the density of the luminance values ​​in a cluster, the greater the slope of the adapted tone mapping curve for that cluster.

[0099] In step 110 shown in FIG. 1, the adapted tone mapping curve resulting from step 109 is smoothed by applying Berstein-Bézier polynomials.

[0100] 4 illustrates schematically the result of steps 109 and 110, which correspond to step e) of the method according to the invention as described above. To this end, FIG. 4 shows an exemplary initial tone mapping curve 1. The initial tone mapping curve 1 is based on the Reinhard global tone mapping operator as described above and is plotted against the HDR input luminance Y on the horizontal axis 61. in , the LDR output luminance Y on the vertical axis 62 outFor each of the three clusters represented by their respective centroids c1, c2, c3, an adapted tone mapping curve 5 is generated by adapting the slope of the initial tone mapping curve 1 in the respective adaptation region aa1, aa2, aa3 of the corresponding cluster depending on the density of the luminance values ​​in the respective adaptation region aa1, aa2, aa3. In this exemplary embodiment, this is done so that the higher the density of the luminance values ​​in a cluster, as represented by the density of the luminance values ​​in the corresponding adaptation region aa1, aa2, aa3, the greater the slope of the adapted tone mapping curve 5 for that cluster.

[0101] As can be seen in Figure 4, within each of the adaptive regions aa1, aa2, and aa3, the slope of the adapted tone mapping curve 5 is greater than the slope of the initial tone mapping curve 1. This is due to the relatively high concentration of luminance values ​​around the cluster centroids c1, c2, and c3. To illustrate these increased slopes of the adapted tone mapping curve 5, the average slopes within each adaptive region aa1, aa2, and aa3 for both the initial tone mapping curve 1 and the adapted tone mapping curve 5 are shown schematically in Figure 4. For each of the three adaptive regions aa1, aa2, and aa3, the average slope m of the adapted tone mapping curve 5 is a1 , m a2 , m a3 is the average gradient m of the initial tone mapping curve 1 i1 , m i2 , m i3 It is found to be significantly larger than

[0102] This increase in the slope of the tone mapping curve around the cluster centroids c1, c2, c3, i.e., in those parts of the luminance range where the concentration of luminance values ​​in the input image is particularly high, results in improved quality of the generated LDR output image, since the local contrast and associated image details of the input image can be effectively preserved.

[0103] Referring again to FIG. 1, in step 111, the LDR output image is generated by applying the adapted tone mapping curve 5 to the luminance component Y and chrominance components Cb and Cr of the HDR input image.

[0104] Finally, in step 112, the resulting chrominance components of the output image are desaturated by applying a desaturation scaling factor to the chrominance components.

[0105] Steps 111 and 112 shown in FIG. 1 therefore correspond to step f) of the method according to the invention as described above.

[0106] Figure 5 illustrates an exemplary image processing system 211. The image processing system 211 includes an HDR image sensor 201, which in the exemplary embodiment of Figure 5 is an HDR video camera. Additionally, the image processing 211 of Figure 5 includes a display unit 209, which in this exemplary embodiment is a conventional LDR display.

[0107] 5 further comprises an electronic device 203, which in this exemplary embodiment is an image processing unit. The image processing unit 203 has a data processing unit 205 and a memory 207 for storing image data. The data processing unit 205 can be, for example, a suitably programmed microprocessor, a digital signal processor (DSP), a graphics processing unit (GPU), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc. The data processing unit 205 reads from and writes to the memory 207.

[0108] The HDR video camera 201 generates a sequence of HDR input images. The HDR video camera 201 is directly or indirectly connected to the image processing unit 203, so that the image processing unit 203 can read the HDR input images generated by the video camera 201. Each HDR input image read by the image processing unit 203 can be stored in the memory 207.

[0109] The image processing unit 203 is adapted to perform the method as described above for tone mapping an HDR input image to an LDR output image.

[0110] After tone mapping the HDR input image to an LDR output image, i.e., after generating the LDR output image as described above, the generated LDR output image can be sent to a display unit 209, where the output image can be displayed.

[0111] This procedure can be repeated for each HDR input image of the HDR video sequence generated by the HDR video camera 201. This results in the generation of an LDR output video sequence, which is a sequence of LDR output images. The LDR output video sequence can be displayed on the display unit 209.

[0112] Additionally or alternatively, the output images and / or the output video sequence may be stored in a memory and / or stored in a data storage unit and / or transmitted via a data transmission link.

[0113] [List of references] 1. Initial Tone Mapping Curve 3 Luminance Histogram 5. Adaptive Tone Mapping Curve 51 Brightness histogram horizontal axis 52 Brightness histogram vertical axis 61 Tone Mapping Curve Horizontal Axis 62 Tone Mapping Curve Vertical Axis 101~112 steps 201 HDR video camera (image sensor) 203 Image Processing Unit (Electronic Device) 205 Data Processing Unit 207 Memory 209 Display Unit 211 Image Processing System aa1, aa2, aa3 adaptive regions b l1 , b l2 , b l3 Lower Boundary b u1 , b u2 , b u3 upper boundary c1, c2, c3 center of gravity d i Distance between the centroids of cluster i and adjacent cluster i m a1 , m a2 , m a3 The gradient of the adapted tone mapping curve m i1 , m i2 , m i3 Initial tone mapping curve gradient The inventions described in the claims of the present application as originally filed are set forth below. [C1] 1. A method for tone mapping a high dynamic range input image to a low dynamic range output image, the method comprising at least: a) obtaining a luminance component of the input image; b) obtaining an initial tone mapping curve (1), which is a global tone mapping curve; c) obtaining a luminance histogram (3) representing the luminance distribution of the input image; d) determining a plurality of clusters of the luminance distribution from the luminance histogram (3), wherein each cluster has a centroid (c 1 、c 2 、c 3 ) e) For each cluster, the gradient (m i1 、m i2 、m i3 ) to generate an adapted tone mapping curve (5), wherein the higher the density of the luminance values ​​in the cluster, the lower the slope (m a1 、m a2 、m a3 ) becomes larger, f) generating the output image by applying the adapted tone mapping curve (5) to at least the luminance component of the input image; A method comprising the step of: [C2] Step c) - compressing said input image; - obtaining the luminance histogram (3) from the compressed input image; The method according to claim 1, comprising: [C3] The method according to C2, characterized in that the input image is compressed by applying a global tone mapping curve, in particular the initial tone mapping curve (1). [C4] The method according to any one of C1 to C3, characterized in that the initial tone mapping curve (1) is based on the global tone mapping operator according to Reinhard. [C5] 5. The method of any one of C1 to C4, wherein in step d) the plurality of clusters are determined by k-means clustering. [C6] Step d) separates two or more clusters based on the distance between these clusters and / or the centroids (c 1 、c 2 、c 3 6. The method of any one of C1 to C5, characterized in that it comprises merging depending on whether the distance is less than a threshold. [C7] The concentration of luminance values ​​in the cluster is at least the amount of variation in the luminance values ​​in the cluster, and / or In particular, the number of luminance values ​​in that cluster relative to the total number of luminance values ​​in said luminance histogram (3). 7. The method according to any one of claims 1 to 6, wherein the value is determined based on: [C8] In step e), for each cluster, the gradient (m i1 、m i2 、m i3 ) is the adaptive region (aa 1 , aa 2 , aa 3 ) according to the density of the luminance values ​​within the adaptive region (aa 1 , aa 2 , aa 3 ) and adapted within the adapted region (aa 1 , aa 2 , aa 3 ) is the centroid (c 1 、c 2 、c 3 8. The method of any one of C1 to 7, characterized in that the range of luminance values ​​around [C9] The adaptive region (aa 1 , aa 2 , aa 3 ) is determined by the size and / or position of the lower boundary (b l1 、b l2 、b l3 ) and the upper boundary (bu1 、b u2 、b u3 ) and the lower boundary (b l1 、b l2 、b l3 ) and / or the upper boundary (b u1 、b u2 、b u3 )teeth, - the centroid (c 1 、c 2 、c 3 ), and / or - the estimated or actual variation of the luminance values ​​in each of the clusters; and / or - the centroids of the different clusters (c 1 、c 2 、c 3 ), in particular, the distance between the centroids (c 1 、c 2 、c 3 ) and the centroid of the adjacent cluster (c 1 、c 2 、c 3 The method according to claim 8, wherein the distance between the first and second nodes is determined according to the distance between the first and second nodes. [C10] Step e) is determining a plurality of luminance zones, each luminance zone covering a portion of the luminance range, and adjusting the gradient (m) of the initial tone mapping curve (1) by adapting its output value for each luminance zone. i1 、m i2 、m i3 10. The method of any one of C1 to C9, comprising adapting [C11] 11. The method of any one of claims 1 to 10, wherein step a) comprises obtaining a luminance component and a plurality of chrominance components of the input image, and step f) comprises applying the adapted tone mapping curve (5) to the luminance component and the plurality of chrominance components. [C12] The method of claim 11, wherein step f) further comprises desaturating the resulting chrominance components of the output image, in particular by applying a desaturation scaling factor to the chrominance components. [C13] The step e) calculates the gradient (m i1 、m i2 、m i3 13. The method according to any one of claims 1 to 12, characterized in that after adapting the tone mapping curve, the resulting adapted tone mapping curve is smoothed, in particular by applying Berstein and / or Berstein-Bézier polynomials and / or Bézier curves. [C14] A computer program comprising program code means adapted to perform the method according to any one of C1 to C13, when said computer program is run on a computer. [C15] An electronic device (203) adapted to perform the method according to any one of C1 to C13.

Claims

1. 1. A method for tone mapping a high dynamic range input image to a low dynamic range output image, the method comprising at least: a) obtaining a luminance component of the input image; b) obtaining an initial tone mapping curve (1) which is a global tone mapping curve; c) obtaining a luminance histogram (3) representing the luminance distribution of the input image; d) determining a plurality of clusters of the luminance distribution from the luminance histogram (3), wherein each cluster has a centroid (c １ , c ２ , c ３ ) e) For each cluster, the slope (m ｉ１ , m ｉ２ , m ｉ３ ) to generate an adapted tone mapping curve (5), wherein the higher the density of the luminance values ​​in the cluster, the lower the slope (m ａ１ , m ａ２ , m ａ３ ) becomes larger, f) generating the output image by applying the adapted tone mapping curve (5) to at least the luminance component of the input image; comprising the step of: wherein step a) comprises obtaining a luminance component and a plurality of chrominance components of the input image, and step f) comprises applying the adapted tone mapping curve (5) to the luminance component and the plurality of chrominance components.

2. Step c) - compressing said input image; - obtaining said luminance histogram (3) from said compressed input image; The method of claim 1 , comprising:

3. The method of claim 2 , wherein the input image is compressed by applying a global tone mapping curve.

4. Method according to any one of claims 1 to 3, characterized in that the initial tone mapping curve (1) is based on the global tone mapping operator according to Reinhard.

5. 5. The method according to claim 1, wherein in step d) the plurality of clusters are determined by k-means clustering.

6. Step d) separates two or more clusters based on the distance between these clusters and / or the centroids (c １ , c ２ , c ３ 6. The method of claim 1, further comprising merging depending on:

7. The concentration of luminance values ​​in the cluster is at least the amount of variation in the luminance values ​​in the cluster, and / or The number of luminance values ​​in that cluster 7. The method according to claim 1, wherein the distance is determined based on:

8. In step e), for each cluster, the gradient (m ｉ１ , m ｉ２ , m ｉ３ ) is the adaptive region (aa １ , aa ２ , aa ３ ) depending on the density of the luminance values ​​within the adaptive region (aa １ , aa ２ , aa ３ ) and adapted within the adapted region (aa １ , aa ２ , aa ３ ) is the centroid (c １ , c ２ , c ３ 8. The method of claim 1, wherein the range of luminance values ​​includes a range of luminance values ​​around .

9. The adaptive region (aa １ , aa ２ , aa ３ ) is determined by the size and / or position of the lower boundary (b ｌ１ , b ｌ２ , b ｌ３ ) and the upper boundary (b ｕ１ , b ｕ２ , b ｕ３ ) and the lower boundary (b ｌ１ , b ｌ２ , b ｌ３ ) and / or the upper boundary (b ｕ１ , b ｕ２ , b ｕ３ )teeth, - the centroid (c １ , c ２ , c ３ ), and / or the estimated or actual variation of the luminance values ​​in the respective clusters, and / or - the centroids of the different clusters (c １ , c ２ , c ３ 9. The method of claim 8, wherein the distance between the first and second electrodes is determined in accordance with the distance between the first and second electrodes.

10. Step e) is determining a plurality of luminance zones, each luminance zone covering a portion of the luminance range, and adjusting the gradient (m) of the initial tone mapping curve (1) by adapting its output value for each luminance zone. ｉ１ , m ｉ２ , m ｉ３ 10. The method according to claim 1, further comprising: adapting the

11. 10. The method of claim 1, wherein step f) further comprises desaturating the resulting chrominance components of the output image.

12. The step e) calculates the gradient (m ｉ１ , m ｉ２ , m ｉ３ 12. The method of claim 1, further comprising adapting the tone mapping curve (a) to the input image (b), followed by smoothing the resulting adapted tone mapping curve.

13. A computer program comprising program code means adapted to perform the method of any one of claims 1 to 12 when said computer program is run on a computer.

14. An electronic device (203) adapted to perform the method according to any one of claims 1 to 12.

Images