Global tone mapping for HDR images with histogram gaps

Abstract

Techniques for improving image quality of an HDR image by increasing contrast in dark portions of the image while retaining bright portions of the image. The method retains an input dynamic range. An image having a luminance histogram gap between a main portion having low luminance and a small portion having high luminance. A first tone mapping curve is determined for a low brightness portion of an image. A second tone mapping curve from a selected point on the first tone mapping curve to a maximum luminance level of the input image is determined. A final tone mapping curve is generated, the final tone mapping curve including a first tone mapping curve from the minimum luminance input to the selected point and a second tone mapping curve from the selected point to the maximum luminance level. The method can improve the overall image quality and contrast.

Description

Technical Field

[0001] The present disclosure relates generally to image processing, and more particularly to tone mapping for images with histogram gaps. Background Art

[0002] Most modern cameras produce high dynamic range (HDR) images. HDR images are often subjected to a tone mapping transformation in an attempt to improve the recognizability of objects in the image scene. However, in many instances, HDR images can have one or more very bright pixel regions, while the rest of the image is very dark. When tone mapping is performed on a dark image with bright regions, there is little improvement in image quality or the appearance of objects in the image scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] The embodiments will be readily understood by the following detailed description in conjunction with the accompanying drawings. To aid this description, similar reference numerals denote similar structural elements. In the figures of the accompanying drawings, the embodiments are illustrated by way of example and not by way of limitation.

[0004] Figure 1 An example overview of an image processing framework that may be used for calibration and / or training according to various embodiments is shown.

[0005] Figure 2 Examples of image processing according to various embodiments are shown.

[0006] Figures 3A-3E is an example diagram illustrating a luminance histogram and a tone mapping curve according to various embodiments.

[0007] Figure 4 A flow chart of a method for tone mapping according to various embodiments is shown.

[0008] Figure 5 A high-level flow chart of a method for tone mapping according to various embodiments is shown.

[0009] Figure 6 A DNN system according to various embodiments is shown.

[0010] Figure 7 is a block diagram of an example computing device in accordance with various embodiments. DETAILED DESCRIPTION

[0011] Overview

[0012] HDR images are typically subjected to a tone mapping transformation in an attempt to improve the recognizability and image quality of objects in the image scene. However, in many instances, an HDR image may have one or more small areas of pixels that are very bright, while the rest of the image is very dark. This occurs very frequently, for example, in scenes with a bright sky, in scenes with bright lights, or in scenes with windows filled with sunlight. In such images, the interesting parts of the image are typically in the dark parts of the image, while the bright parts of the image are less important. From a consumer's perspective, the goal of such images is to improve the quality of the interesting parts of the image (the dark parts) as much as possible, while not overexposing (or eliminating) the bright parts of the image. However, when a general tone mapping transformation is performed on a dark image with bright areas, there is little improvement in the image quality or the appearance of objects in the image scene.

[0013] When image pixel brightness is plotted in a histogram, a dark image with one or more bright areas may have a large gap in the histogram between a small number of very bright pixels and a large number of dark pixels (representing the rest of the image). Using a pixel brightness histogram, dark and bright portions of an image can be identified. Techniques are provided herein to improve image quality by increasing contrast in dark portions of an image while preserving bright portions and preserving input dynamic range. In particular, an image is identified that has a luminance histogram gap between a major portion with low brightness and a minor portion with high brightness. A first tone mapping curve is determined for the low brightness portion of the image. A second tone mapping curve is determined from a selected point on the first tone mapping curve to the maximum brightness level of the input image. A final tone mapping curve is generated, comprising the first tone mapping curve from the minimum brightness input to the selected point and the second tone mapping curve from the selected point to the maximum brightness level. In some examples, this method can improve image quality and contrast for intentionally captured faces, as faces are typically a dominant portion of an image.

[0014] For purposes of explanation, specific numbers, materials, and configurations are set forth to provide a thorough understanding of the illustrative implementations. However, it will be apparent to those skilled in the art that the present disclosure may be practiced without these specific details, or / and with only some of the described aspects. In other instances, well-known features are omitted or simplified to avoid obscuring the illustrative implementations.

[0015] In addition, reference is made to the accompanying drawings which form a part hereof, and in which are shown by way of illustration embodiments that can be implemented. It should be understood that other embodiments can be utilized, and structural or logical changes can be made without departing from the scope of the present disclosure. Therefore, the following detailed description should not be construed in a limiting sense.

[0016] The various operations may be described as a plurality of discrete actions or operations in a manner that is most helpful in understanding the claimed subject matter. However, the order of description should not be interpreted as implying that these operations are necessarily sequentially related. In particular, these operations may not be performed in the order presented. The described operations may be performed in an order different from that of the described embodiment. In other embodiments, various other operations may be performed, or the described operations may be omitted.

[0017] For the purposes of this disclosure, the phrase "A and / or B" or the phrase "A or B" means (A), (B), or (A and B). For the purposes of this disclosure, the phrase "A, B and / or C" or the phrase "A, B or C" means (A), (B), (C), (A and B), (A and C), (B and C), or (A, B and C). When used in reference to a measurement range, the term "between" includes the endpoints of the measurement range.

[0018] The description uses the phrases "in one embodiment" or "in an embodiment", each of which may refer to one or more of the same or different embodiments. As used with respect to the embodiments of the present disclosure, the terms "including", "comprising", "having" and the like are synonymous. The present disclosure may use perspective-based descriptions such as "upper", "lower", "top", "bottom" and "side" to explain various features of the drawings, but these terms are merely for ease of discussion and do not imply desired or required orientations. The drawings are not necessarily drawn to scale. Unless otherwise stated, the use of ordinal adjectives such as "first", "second", and "third" to describe common objects merely indicates that different instances of similar objects are being referred to and is not intended to imply that the objects so described must be in a given order in time, space, hierarchy, or in any other manner.

[0019] In the following detailed description, various aspects of the illustrative implementations will be described using terms commonly employed by those skilled in the art to convey the substance of their work to others skilled in the art.

[0020] The terms "substantially," "close," "approximately," "nearly," and "about" generally refer to within + / - 20% of a target value for an input operand based on specific values ​​as described herein or as known in the art. Similarly, terms indicating the orientation of various elements, such as "coplanar," "perpendicular," "orthogonal," "parallel," or any other angle between elements, generally refer to within + / - 5-20% of a target value for an input operand based on specific values ​​as described herein or as known in the art.

[0021] Furthermore, the terms "comprise," "comprising," "include," "including," "have," "having," or any other variations thereof are intended to cover a non-exclusive inclusion. For example, a method, process, apparatus, or system that comprises a list of elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such method, process, apparatus, or system. Furthermore, the term "or" refers to an inclusive "or" rather than an exclusive "or."

[0022] The systems, methods, and devices of the present disclosure each have several innovative aspects, no single one of which is solely responsible for all of the desirable attributes disclosed herein. The details of one or more implementations of the subject matter described in this specification are set forth in the following description and drawings.

[0023] Example Tone Mapping Framework

[0024] Figure 1 An example overview of a tone mapping framework 100 that can be used to process an image with a high dynamic range according to various embodiments is shown. In some examples, the tone mapping framework 100 can be used to process any image, and a determination that the image has a high dynamic range can be determined at the image processing unit 104. When the image processing unit 104 determines that the image has a high dynamic range, the image can be processed using global tone mapping as described herein. In some examples, the tone mapping framework 100 is as described with respect to Figure 6 In some examples, the tone mapping framework 100 is as described with respect to Figure 7 A portion of computing device 700 is depicted.

[0025] like Figure 1As shown in , the image processing unit 104 receives the input image 102. The luminance determination module 108 determines the luminance of the input image. In particular, the luminance determination module 108 determines the brightness of each pixel. In various examples, a luminance histogram can show how many pixels fall into each brightness level. The image histogram module 106 can generate the luminance histogram, and the image histogram module 106 determines whether there are gaps in the luminance histogram. Generally, if most pixels are located at the lower end of the histogram (and have low brightness levels), while a small number of pixels are distributed at the high end of the histogram (and have high brightness levels), with few or no pixels in between, then the luminance histogram has gaps. In some examples, the small number of pixels at the high end of the histogram represents approximately 2% of the pixels in the input image 102, or less than 2% of the pixels in the input image 102. In some examples, the small number of pixels at the high end of the histogram represents less than approximately 5% of the pixels in the input image 102, or less than approximately 10% of the pixels in the input image 102. In various examples, the image having luminance histogram gaps is an HDR image.

[0026] The tone mapping module 110 determines the output brightness of each pixel in the output image 112 by determining a luminance gain, where the luminance gain is a value by which the measured input luminance level of each pixel is multiplied. In general, the luminance gain is a function of the input luminance and the luminance histogram. In particular, when the image histogram module 106 determines that there are gaps in the luminance histogram, the tone mapping module 110 generates a global tone mapping curve (i.e., a luminance gain curve) for tone mapping that provides increased contrast in dark portions of the image while preserving bright portions of the image and the input dynamic range. Figure 2 、 3A 3E, 4, and 5 describe global tone mapping in more detail. When the image histogram module 106 does not find a definable gap in the luminance histogram, the tone mapping module 110 can use a conventional tone mapping curve and perform typical tone mapping. The tone mapping module 110 generates an output image 112 in which the output luminance of each pixel is determined using the selected tone mapping curve. The image processing unit 104 outputs the output image 112.

[0027] Example global tone mapping output

[0028] Figure 2 An example 200 of an image processed with a global tone mapping method according to various embodiments is shown. In particular, Figure 2The scene captured by camera 215 in FIG. 2 is an indoor scene that includes a person in the foreground and a window in the background. Sunlight is shining through window 212. The sunlight causes very bright areas 222 in captured image 220. As shown in example 200, captured image 220 appears very dark, except for bright portion 222 of the image representing the window. In various examples, the luminance histogram of captured image 220 has large gaps, where most pixels have low luminance, and a small portion of pixels (from bright portion 222 of captured image 220) have very high luminance, with few or no pixels having luminance in the gaps in between.

[0029] Figure 3A is an example luminance histogram 300 showing histogram gaps 302 according to various embodiments. In some examples, Figure 3A is a luminance histogram 300 representing the brightness of pixels in the captured image 220 .

[0030] Figure 3B An example 310 of a typical tone mapping curve (i.e., luminance gain curve) 312 is shown according to various embodiments. The typical tone mapping curve 312 can be used to adjust the brightness of pixels in an image to increase contrast. However, as Figure 3B As shown in , when a typical tone mapping curve 312 is used on a luminance histogram having large gaps 302, a large portion of the tone mapping curve 312 is applied to luminance levels for which no input pixels have the selected luminance level. In particular, a portion 314 of the luminance gain curve 312 is not applied to any pixels because no input pixels fall within the input luminance values ​​of that portion of the tone mapping curve 312. Therefore, when the tone mapping module applies the tone mapping curve 312 to the captured image 220, the dark portions of the output image 230 are only slightly brighter and still appear too dark.

[0031] However, as described herein, using information about the histogram gaps 302 , the image processing unit 204 can perform global tone mapping on the captured image 220 to generate an output image 240 having increased contrast in dark portions of the image while preserving bright portions 242 of the image and the input dynamic range.

[0032] Figure 3C An example 320 of an auxiliary TM curve 322 is shown according to various embodiments. Figure 3C, the majority of image pixels in luminance histogram 300 are below threshold 304. Threshold 304 can have a percentile value PT that represents the percentage of input image pixels having a measured input luminance level below threshold 304. In some examples, percentile value PT is adjustable and predetermined, and threshold 304 is located at the highest measured input luminance level of the darkest PT percentage of input image pixels. In some examples, for any given input image, a luminance level for threshold 304 is determined on the x-axis. In some examples, a neural network can be used to determine an optimal percentile value PT and / or luminance level for threshold 304.

[0033] Generates an auxiliary TM curve (TM) for input pixels with brightness between 0 and threshold 304 α ) 322, where the auxiliary TM curve 322 has a maximum output at the threshold 304. Although the auxiliary TM curve 322 increases the dynamic range of low-luminance pixels, the auxiliary TM curve 322 is not used for the entire image because pixels above the threshold 304 will reach luminance saturation and thus the input image will be clipped.

[0034] On the contrary, Figure 3D As shown in , a connection point CP is defined. In particular, an output brightness level (on the y-axis) is selected and / or predetermined for the connection point. In some examples, the selected output brightness level of the connection point is between about 0.5 and about 0.7 (on the y-axis). Figure 3D An example 330 of a connection point 332 on the auxiliary TM curve 322 is shown according to various embodiments. At the connection point CP, the auxiliary TM curve reaches a selected output brightness level (Y CP The corresponding measured input brightness level (X) on the x-axis can be determined based on the auxiliary TM curve. CP ). In particular, point X CP is determined so that:

[0035] TM α (X CP )=Y CP (1)

[0036] From point (X CP , Y CP )320, the brightness level is higher than X CP The tail curve (F(x)) 330 is generated for input pixels with brightness levels above X CP For input pixels of , using the tail curve (F(x)) 330 avoids clipping (ie, overexposure) at higher brightness levels while also compressing less important data in the bright portions of the image. Figure 3EShown is an example 340 of a full tone mapping curve 342 for global tone mapping as described herein, according to various embodiments.

[0037] According to some implementations, to generate the full tone mapping curve 342, the auxiliary TM curve is determined at the connection point (X CP , Y CP ). In some examples, the slope can be determined as:

[0038] SL=TM′ α (X CP ) (2)

[0039] The tail curve is also determined:

[0040] TM tail (x)=F(x,t,g) (3)

[0041] In some examples, F(x, t, g) is defined using the following formula and some constant parameters t and g:

[0042]

[0043] In equation (4), X END is the point where the original input signal ends. That is, X END is the last bin of the luminance histogram and has the maximum luminance level. According to various examples, the parameters t and g are defined such that the derivative F'(x, t, g) is equal to the slope SL:

[0044] F′(x,t,g)=SL (5)

[0045] In various examples, equation (5) facilitates smoothing of the full tone mapping curve.

[0046] Therefore, using equations (1)-(5), the full tone mapping curve TM(x) is constructed as TM α (x) curve and TM tail (x) Cascade of curves:

[0047]

[0048] The full tone mapping curve TM(x) produces an output image with good contrast while not limiting the dynamic range of the input image data. The full tone mapping curve TM(x) can be used for a global tone mapping transformation of an input image to generate an output image.

[0049] Example method for global tone mapping

[0050] Figure 4is a flow chart illustrating a method 400 for generating a full tone mapping curve TM(x) according to various embodiments. Figure 4 The flowchart shown in describes method 400, but many other methods for global tone mapping may alternatively be used. For example, Figure 4 As another example, some steps may be changed, deleted, or combined. In various examples, method 400 may be performed by an image processing unit (e.g., Figure 1 The image processing unit 104) is implemented.

[0051] At step 410, an input image is received. In some examples, the input image may be an image frame of a video or other series of image frames. In some examples, the input image may be an image frame of a real-time video.

[0052] At step 420, a luminance histogram is generated. In some examples, the luminance histogram is a brightness histogram. A luminance histogram plots the number of pixels at each of a plurality of brightness levels. The brightness levels can vary from brightness level 0 (maximum darkness / minimum brightness) to brightness level 1 (maximum brightness). For each brightness level, the number of pixels of the input image having the selected brightness level is determined to produce a luminance histogram. In various examples, at step 420, the number of pixels at each brightness level is determined without generating an actual luminance histogram.

[0053] At step 430, it is determined whether there are gaps in the luminance histogram. In particular, it is determined whether there are multiple consecutive luminance levels where few or no pixels have those luminance levels. In some examples, the multiple consecutive luminance levels with few or no pixels comprise more than approximately one-third of the luminance scale, in some examples, the multiple consecutive luminance levels comprise approximately half of the luminance scale, and in some examples, the multiple consecutive luminance levels comprise more than approximately half of the luminance scale. Typically, a major portion of an image may have low luminance levels, while a minor portion of the image may have high luminance levels. If there are gaps in the luminance histogram, method 400 proceeds to step 440. If there are no gaps in the luminance histogram, method 400 ends.

[0054] At step 440, a brightness percentile of a major portion of the input image is determined. The brightness percentile PT can be a threshold on the x-axis brightness histogram. In various examples, the brightness percentile is a brightness percentile. In one example, the major portion of the input image is approximately 98% of the pixels and represents the brightness data of approximately 98% of the image. In some examples, this can be expressed as: PT = getBrightnessPercentile(imageData, brTH), where brTH = 98. In other examples, the major portion of the input image is approximately 95% of the pixels and represents the brightness data of approximately 95% of the image, and in some examples, the major portion of the input image is more than 90% of the pixels and represents the brightness data of more than 90% of the image.

[0055] At step 450, an auxiliary tone map TM is generated α , which maps the main portion of the image (representing 98% of the image's luminance data) to the full range luminance output. The main portion of the image includes image pixels having luminances at and / or below the luminance percentile. In some examples, this can be expressed as: TM α =buildTMLUT(imageData,PT). In some examples, the auxiliary tone mapping can be done using a lookup table. In some examples, the auxiliary tone mapping can be generated using any tone mapping technique that produces good luminance contrast.

[0056] At step 460, identify the auxiliary tone mapping TM α Adjustable luminance output target Y CP In some examples, the luma output is normalized to a full range of output values ​​such that the maximum Y value is 1, and the adjustable luma output target Y CP is set to 0.6. In step 470, using Y CP , determine the corresponding brightness input X CP In some examples, based on Assisted Tone Mapping™ α To determine the corresponding brightness input X CP In some examples, the point (X CP , Y CP ) is a point on the auxiliary tone mapping curve. Point (X CP , Y CP ) can be a connection point in the input-output tone mapping space, and the point (X CP , Y CP ) can be the point where the brighter portion of the tone mapping curve connects to the secondary tone mapping curve to generate the full tone mapping curve.

[0057] At step 475, determine the auxiliary tone map TMα At the connection point (X CP , Y CP ) at the gradient SL. The gradient can be the gradient from the left side of the auxiliary tone mapping curve. In various examples, determining the gradient includes: using the auxiliary tone mapping TM α The last two {input, output} pairs of and divide the differences.

[0058] In step 480, the analytical function is defined. In particular, the analytical function F(x, t, g) can be defined as discussed herein with respect to equations (4) and (5). Using the gradient and the connection point (X) from step 475 CP , Y CP ), the value of g can be fixed and the corresponding value of t can be determined. For example, g can be fixed at (g0=0.5) and t0 can be determined using the following formula:

[0059]

[0060] where F' is the first derivative of F. In various examples, the results from the above formulas can be efficiently determined analytically or by iteration.

[0061] In step 485, using the values ​​of t0 and g0 from step 480, for values ​​greater than x CP For any input x, the tail part of the tone mapping curve can be defined as TM tail (x) = F(x, t0, g0). Therefore, as mentioned above, the full tone mapping curve is:

[0062]

[0063] where X END is the point where the original input signal ends.

[0064] Figure 5 is a flow chart illustrating a high-level method 500 for generating a full tone mapping curve TM(x) according to various embodiments. Figure 5 The flowchart shown in describes method 500, but many other methods for global tone mapping may alternatively be used. For example, Figure 5 As another example, some steps may be changed, deleted, or combined. In various examples, method 500 may be performed by an image processing unit (e.g., Figure 1 The image processing unit 104) is implemented.

[0065] At step 510, an input image frame is received from an image sensor. At step 520, a luminance histogram is generated. In various examples, as described above, a luminance histogram plots the number of pixels at each of a plurality of luminance levels. For each luminance level, the number of pixels of the input image having a selected luminance is determined to generate a luminance histogram. In various examples, at step 520, the number of pixels at each luminance level is determined without generating an actual luminance histogram.

[0066] At step 530, it is determined whether there are gaps in the luminance histogram. In particular, it is determined whether there are multiple consecutive luminance levels where few or no pixels have those luminance levels. In various examples, a major portion of the input image frame has a low luminance level, while a minor portion of the input image frame has a high luminance level. A gap represents multiple consecutive luminance levels where few or no pixels have those luminance levels between the major portion and the minor portion of the input image frame.

[0067] At step 540, a first tone mapping curve is generated for the main portion of the input image frame. The first tone mapping curve maps measured input luminance levels to target output luminance levels. Typically, because the main portion of the image has low luminance levels, the first tone mapping curve generates a target output luminance level that is greater than the measured input luminance level for each measured input luminance level, thereby increasing the contrast between the luminance levels of the pixels of the input image frame. In one example, the measured input luminance levels of the main portion vary between 0 (minimum luminance / completely dark) and a luminance level of approximately 0.3 (where the maximum luminance level is 1), and the first tone mapping curve maps measured input luminance levels between 0-0.3 to target output luminance levels between 0 and 1 (0-1 luminance levels).

[0068] At step 550, a selected luma output target is identified, and a point on the first tone mapping curve having the selected luma output target is identified (this is the selected luma output target point). In one example, the selected luma output target can be approximately 0.6 on a 0-1 luma level scale. In some examples, a corresponding measured input luma level on the first tone mapping curve is also identified.

[0069] At step 560, a second tone mapping curve is generated, where the second tone mapping curve is a curve from the selected luma output target point on the first tone mapping curve to the maximum luma level of the luma histogram. Thus, in an example where the selected luma output target is 0.6, a second tone mapping curve is generated from the point on the first tone mapping curve where the selected luma output target is 0.6 to a point on the luma histogram with a maximum luma level of 1. In some examples, the selected luma output target is 0.6, and a second tone mapping curve is generated from the point on the first tone mapping curve where the selected luma output target is 0.6 to a point where the measured input luma level is 1 and the luma output target is also 1.

[0070] At step 570, a full tone mapping curve is generated, wherein the full tone mapping curve includes a first portion of the first tone mapping curve and a second tone mapping curve. In particular, the full tone mapping curve includes a portion of the first tone mapping curve from the minimum luma input to the selected luma output target point identified at step 550, coupled to the second tone mapping curve.

[0071] Example DNN system

[0072] Figure 6 is a block diagram of an example DNN system 600 according to various embodiments. The DNN system 600 trains a DNN for various tasks, including image processing of captured image frames. The DNN system 600 includes an interface module 610, an image processing unit 620, a training module 630, a verification module 640, an inference module 650, and a data store 660. In other embodiments, alternative configurations, different, or additional components may be included in the DNN system 600. Furthermore, the functionality attributed to a component of the DNN system 600 may be accomplished by different components included in the DNN system 600 or in a different system. The DNN system 600 or a component of the DNN system 600 (e.g., the training module 630 or the inference module 650) may include Figure 7 The computing device 700 in FIG.

[0073] The interface module 610 facilitates communication between the DNN system 600 and other systems. As an example, the interface module 610 supports the DNN system 600 in distributing trained DNNs to other systems, such as computing devices configured to apply the DNNs to perform tasks. As another example, the interface module 610 establishes communication between the DNN system 600 and an external database to receive data that can be used to train the DNN or input to the DNN to perform tasks. In some embodiments, the data received by the interface module 610 can have a data structure, such as a matrix. In some embodiments, the data received by the interface module 610 can be an image, a series of images, and / or a video stream.

[0074] The image processing unit 620 performs image processing including tone mapping on the input image. Typically, the image processing unit 620 reviews the input data and performs global tone mapping on the input image. In various examples, the image processing unit 620 identifies an HDR image.

[0075] The training module 630 trains the DNN using a training dataset. In some embodiments, the training dataset used to train the DNN may include one or more images and / or videos, each of which may be a training sample. In some examples, the training module 630 trains the image processing unit 620. The training module 630 may receive real-world video data for processing by the image processing unit 620, as described herein. In some embodiments, the training module 630 may input different data into different layers of the DNN. For each subsequent DNN layer, the input data may be less than that of the previous DNN layer. In some embodiments, a portion of the training dataset may be used to initially train the DNN, while the remainder of the training dataset may be retained as a validation subset used by the validation module 640 to validate the performance of the trained DNN. The portion of the training dataset that does not include the adjustment subset and the validation subset may be used to train the DNN.

[0076] Training module 630 also determines hyperparameters for training the DNN. Hyperparameters are variables that specify the DNN training process. Hyperparameters are distinct from parameters within the DNN (e.g., filter weights). In some embodiments, hyperparameters include variables that determine the DNN's architecture, such as the number of hidden layers. Hyperparameters also include variables that determine how the DNN is trained, such as the batch size and the number of epochs. The batch size defines the number of training samples to be processed before updating the DNN's parameters. The batch size is equal to or less than the number of samples in the training dataset. The training dataset can be divided into one or more batches. The number of epochs defines the number of times the entire training dataset is passed forward and backward through the network. The number of epochs defines the number of times the deep learning algorithm processes the entire training dataset. An epoch means that every training sample in the training dataset has had an opportunity to update the DNN's internal parameters. An epoch can include one or more batches. The number of epochs can be 1, 10, 50, 100, or even larger.

[0077] The training module 630 defines the architecture of the DNN based on, for example, certain hyperparameters. The DNN architecture includes an input layer, an output layer, and multiple hidden layers. The DNN input layer may include tensors (e.g., multidimensional arrays) that specify attributes of the input image, such as the height, width, and depth of the input image (e.g., the number of bits specifying the color of a pixel in the input image). The output layer includes labels for objects in the input layer. Hidden layers are layers between the input and output layers. Hidden layers include one or more convolutional layers and one or more other types of layers, such as pooling layers, fully connected layers, normalization layers, softmax or logistic layers, and the like. The convolutional layers of the DNN abstract the input image into feature maps, which are represented by tensors specifying the height, width, and channels of the feature maps (e.g., a red, green, and blue image includes three channels). Pooling layers are used to reduce the spatial volume of the input image after convolution. They are used between two convolutional layers. Fully connected layers involve weights, biases, and neurons. They connect neurons in one layer to neurons in another layer. It is used to classify images between different categories through training.

[0078] In the process of defining the architecture of the DNN, the training module 630 also adds an activation function to the hidden layer or output layer. The activation function of a layer transforms the weighted sum of the input of the layer into the output of the layer. The activation function can be, for example, a rectified linear unit activation function, a tangent activation function, or other types of activation functions.

[0079] After the training module 630 defines the architecture of the DNN, the training module 630 inputs the training dataset into the DNN. The training dataset includes multiple training samples. An example of a training dataset includes a series of images from a video stream. The internal parameters include the weights of the filters in the convolutional layer of the DNN. In some embodiments, the training module 630 uses a cost function to minimize the difference. The training module 630 can train the DNN for a predetermined number of epochs. The number of epochs is a hyperparameter that defines the number of times the deep learning algorithm will process the entire training dataset. One epoch means that every sample in the training dataset has had an opportunity to update the internal parameters of the DNN. After the training module 630 completes the predetermined number of epochs, the training module 630 can stop updating the parameters in the DNN. The DNN with updated parameters is called a trained DNN.

[0080] The validation module 640 validates the accuracy of the trained DNN. In some embodiments, the validation module 640 inputs samples from the validation dataset into the trained DNN and uses the output of the DNN to determine model accuracy. In some embodiments, the validation dataset may be formed from some or all samples in the training dataset. Furthermore, the validation dataset may include additional samples in addition to the samples in the training set, or alternatively, the validation dataset may include additional samples in addition to the samples in the training set. In some embodiments, the validation module 640 may determine an accuracy score that measures the precision, recall, or a combination of precision and recall of the DNN. The validation module 640 may determine the accuracy score using the following metrics: precision = TP / (TP+FP) and recall = TP / (TP+FN), where precision can be the number of correct predictions (TP or true positives) made by the reference classification model out of the total number of its predictions (TP+FP or false positives), and recall can be the number of correct predictions (TP) made by the reference classification model out of the total number of objects that actually possess the attribute (TP+FN or false negatives). F-score (F-score = 2*PR / (P+R)) unifies precision and recall into a single metric.

[0081] The validation module 640 can compare the accuracy score to a threshold score. In instances where the validation module 640 determines that the accuracy score of the enhanced model is below the threshold score, the validation module 640 instructs the training module 630 to retrain the DNN. In one embodiment, the training module 630 can iteratively retrain the DNN until a stopping condition occurs, such as an accuracy measurement indicating that the DNN is sufficiently accurate, or a number of training rounds have occurred.

[0082] The reasoning module 650 applies the trained or verified DNN to perform a task. The reasoning module 650 can run the reasoning process of the trained or verified DNN. For example, the reasoning module 650 can input real-world data into the DNN and receive the output of the DNN. The output of the DNN can provide a solution to the task for which the DNN is trained. The reasoning module 650 can aggregate the output of the DNN to generate the final result of the reasoning process. In some embodiments, the reasoning module 650 can distribute the DNN to other systems, such as computing devices that communicate with the DNN system 600, so that other systems can apply the DNN to perform tasks. The distribution of the DNN can be accomplished through the interface module 610. In some embodiments, the DNN system 600 can be implemented in servers such as cloud servers, edge services, and the like. The computing device can be connected to the DNN system 600 via a network. Examples of computing devices include edge devices.

[0083] The data storage 660 stores data received, generated, used, or otherwise associated with the DNN system 600. For example, the data storage 660 stores videos processed by the image processing unit 620 or used by the training module 630, the verification module 640, and the inference module 650. The data storage 660 may also store other data generated by the training module 630 and the verification module 640, such as hyperparameters used to train the DNN, internal parameters of the trained DNN (e.g., values ​​of adjustable parameters of activation functions, such as fractional adaptive linear units (FALUs)), etc. Figure 6 In some embodiments, the data store 660 is a component of the DNN system 600. In other embodiments, the data store 660 may be external to the DNN system 600 and communicate with the DNN system 600 over a network.

[0084] Example computing device

[0085] Figure 7 is a block diagram of an example computing device 700 according to various embodiments. In some embodiments, the computing device 700 may be used to Figure 6 At least a portion of the deep learning system 600 in. Multiple components in Figure 7 700, but any one or more of these components may be omitted or duplicated as appropriate for the application. In some embodiments, some or all of the components included in computing device 700 may be attached to one or more motherboards. In some embodiments, some or all of these components are fabricated onto a single system-on-chip (SoC) die. Additionally, in various embodiments, computing device 700 may not include a processor. Figure 7 , computing device 700 may include one or more components shown in , but computing device 700 may include interface circuitry for coupling to one or more components. For example, computing device 700 may not include display device 706, but may include display device interface circuitry (e.g., connectors and driver circuitry) to which display device 706 may be coupled. In another set of examples, computing device 700 may not include video input device 718 or video output device 708, but may include video input or output device interface circuitry (e.g., connectors and supporting circuitry) to which video input device 718 or video output device 708 may be coupled.

[0086] The computing device 700 may include a processing device 702 (e.g., one or more processing devices). The processing device 702 processes electronic data from registers and / or memory to transform the electronic data into other electronic data that can be stored in registers and / or memory. The computing device 700 may include a memory 704, which itself may include one or more memory devices, such as volatile memory (e.g., DRAM), non-volatile memory (e.g., read-only memory (ROM)), high bandwidth memory (HBM), flash memory, solid-state memory, and / or a hard drive. In some embodiments, the memory 704 may include a memory that shares a die with the processing device 702. In some embodiments, the memory 704 includes one or more non-transitory computer-readable media that store executable instructions for occupancy mapping or collision detection, for example, in combination with the above. Figure 4 and 5 Methods 400 and 500 described, or by Figure 1 Some operations performed by the image processing unit 104, or by Figure 6 Some operations performed by the DNN system 600. The instructions stored in one or more non-transitory computer-readable media can be executed by the processing device 702.

[0087] In some embodiments, computing device 700 may include a communication chip 712 (e.g., one or more communication chips). For example, communication chip 712 may be configured to manage wireless communications for transmitting data to and from computing device 700. The term "wireless" and its derivatives may be used to describe circuits, devices, systems, methods, techniques, communication channels, and the like that may use modulated electromagnetic radiation to transmit data through a non-solid medium. The term does not imply that the associated devices do not contain any wiring, although in some embodiments they may not.

[0088] The communication chip 712 can implement any of a variety of wireless standards or protocols, including but not limited to Institute of Electrical and Electronics Engineers (IEEE) standards, including Wi-Fi (IEEE 802.10 series), IEEE 802.16 standards (e.g., IEEE 802.16-2005 amendment), Long Term Evolution (LTE) project and any amendments, updates and / or revisions (e.g., LTE-Advanced project, Ultra Mobile Broadband (UMB) project (also known as "3GPP2"), etc.). Broadband wireless access (BWA) networks that comply with IEEE 802.16 are commonly referred to as WiMAX networks. WiMAX is an acronym standing for Worldwide Interoperability for Microwave Access, which is a certification mark for products that pass conformance and interoperability testing of the IEEE 802.16 standard. The communication chip 712 can operate according to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS), High Speed ​​Packet Access (HSPA), Evolved HSPA (E-HSPA), or LTE networks. The communication chip 712 can operate according to Enhanced Data for GSM Evolution (EDGE), GSM EDGE Radio Access Network (GERAN), Universal Terrestrial Radio Access Network (UTRAN), or Evolved UTRAN (E-UTRAN). The communication chip 712 can operate according to Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Digital Enhanced Cordless Telecommunications (DECT), Evolution Data Optimized (EV-DO) and its derivatives, as well as any other wireless protocols designated as 3G, 4G, 5G and higher. In other embodiments, the communication chip 712 can operate according to other wireless protocols. The computing device 700 may include an antenna 722 to facilitate wireless communication and / or receive other wireless communications (e.g., AM or FM radio transmissions).

[0089] In some embodiments, the communication chip 712 can manage wired communications, such as electrical, optical, or any other suitable communication protocol (e.g., Ethernet). As described above, the communication chip 712 can include multiple communication chips. For example, the first communication chip 712 can be dedicated to short-range wireless communications such as Wi-Fi or Bluetooth, while the second communication chip 712 can be dedicated to longer-range wireless communications such as Global Positioning System (GPS), EDGE, GPRS, CDMA, WiMAX, LTE, EV-DO, or others. In some embodiments, the first communication chip 712 can be dedicated to wireless communications, while the second communication chip 712 can be dedicated to wired communications.

[0090] Computing device 700 may include battery / power circuitry 714. Battery / power circuitry 714 may include one or more energy storage devices (e.g., batteries or capacitors) and / or circuitry for coupling components of computing device 700 to an energy source separate from computing device 700 (e.g., AC line power).

[0091] Computing device 700 may include a display device 706 (or corresponding interface circuitry, as described above). For example, display device 706 may include any visual indicator, such as a heads-up display, a computer monitor, a projector, a touch screen display, a liquid crystal display (LCD), a light-emitting diode display, or a flat-panel display.

[0092] Computing device 700 may include video output device 708 (or corresponding interface circuitry, as described above). For example, video output device 708 may include any device that generates an audible indicator, such as a speaker, headphones, or earphones.

[0093] Computing device 700 may include video input device 718 (or corresponding interface circuitry, as described above). Video input device 718 may include any device that generates a signal representing sound, such as a microphone, a microphone array, or a digital instrument (e.g., an instrument with a Musical Instrument Digital Interface (MIDI) output).

[0094] Computing device 700 may include GPS device 716 (or corresponding interface circuitry, as described above). As is known in the art, GPS device 716 may communicate with a satellite-based system and may receive the location of computing device 700.

[0095] Computing device 700 may include another output device 710 (or corresponding interface circuitry, as described above). Examples of another output device 710 may include an audio codec, a video codec, a printer, a wired or wireless transmitter for providing information to other devices, or another storage device.

[0096] The computing device 700 may include another input device 720 (or corresponding interface circuitry, as described above). Examples of another input device 720 may include an accelerometer, a gyroscope, a compass, an image capture device, a keyboard, a cursor control device (e.g., a mouse, a stylus, a touchpad), a barcode reader, a Quick Response (QR) code reader, any sensor, or a radio frequency identification (RFID) reader.

[0097] The computing device 700 may have any desired form factor, such as a handheld or mobile computer system (e.g., a cellular phone, a smartphone, a mobile internet device, a music player, a tablet computer, a laptop computer, a netbook computer, an ultrabook computer, a personal digital assistant (PDA), an ultra-mobile personal computer, etc.), a desktop computer system, a server or other networked computing component, a printer, a scanner, a monitor, a set-top box, an entertainment control unit, a vehicle control unit, a digital camera, a digital video recorder, or a wearable computer system. In some embodiments, the computing device 700 may be any other electronic device that processes data.

[0098] Selected Examples

[0099] The following paragraphs provide various examples of the embodiments disclosed herein.

[0100] Example 1 provides a computer-implemented method comprising: receiving an input image frame from an imager; generating a luminance histogram; determining that there is a gap in the luminance histogram, wherein the gap is between a main portion of the input image frame having low luminance and a small portion of the input image frame having high luminance; generating a first tone mapping curve for the main portion of the input image frame, wherein the first tone mapping curve maps a measured input luminance level to a target output luminance level; identifying a selected luminance output target point on the first tone mapping curve; generating a second tone mapping curve from the selected luminance output target point to a maximum luminance level of the luminance histogram; and generating a final tone mapping curve, wherein the final tone mapping curve comprises the first tone mapping curve and the second tone mapping curve from a minimum luminance input to the selected luminance output target point.

[0101] Example 2 provides the computer-implemented method of Example 1, further comprising determining a percentile for the principal portion, wherein the percentile represents a first number of pixels in the principal portion compared to a total number of pixels in the input image frame.

[0102] Example 3 provides the computer-implemented method of Example 2, wherein the percentile is greater than 90%.

[0103] Example 4 provides the computer-implemented method according to any of the above examples, further comprising determining a selected luma input level corresponding to the selected luma output target point on the first tone mapping curve.

[0104] Example 5 provides the computer-implemented method according to any of the above examples, further comprising determining a gradient of the first tone mapping curve at the selected luma input level.

[0105] Example 6 provides a computer-implemented method according to any of the above examples, wherein determining that the gap exists in the luminance histogram comprises identifying a pixel-free portion of the luminance histogram, wherein approximately no pixels have a luminance level in the pixel-free portion.

[0106] Example 7 provides a computer-implemented method according to any of the above examples, further comprising generating an output image frame using the final tone mapping curve, wherein the output image frame includes a first output portion corresponding to the main portion of the input image frame, and wherein the first output portion has increased contrast compared to the main portion of the input image frame.

[0107] Example 8 provides one or more non-transitory computer-readable media storing instructions executable to perform operations, the operations comprising: receiving an input image frame from an imager; generating a luminance histogram; determining that there is a gap in the luminance histogram, wherein the gap is between a main portion of the input image frame having low luminance and a small portion of the input image frame having high luminance; generating a first tone mapping curve for the main portion of the input image frame, wherein the first tone mapping curve maps a measured input luminance level to a target output luminance level; identifying a selected luminance output target point on the first tone mapping curve; generating a second tone mapping curve from the selected luminance output target point to a maximum luminance level of the luminance histogram; and generating a final tone mapping curve, the final tone mapping curve comprising the first tone mapping curve and the second tone mapping curve from a minimum luminance input to the selected luminance output target point.

[0108] Example 9 provides one or more non-transitory computer-readable media of Example 8, wherein the operation further includes determining a percentile of the main portion, wherein the percentile represents a first number of pixels in the main portion compared to a total number of pixels in the input image frame.

[0109] Example 10 provides the one or more non-transitory computer-readable media of Example 9, wherein the percentile is greater than 90%.

[0110] Example 11 provides the one or more non-transitory computer-readable media of any above example, the operations further comprising determining a selected luma input level corresponding to the selected luma output target point on the first tone mapping curve.

[0111] Example 12 provides the one or more non-transitory computer-readable media of any above example, the operations further comprising determining a gradient of the first tone mapping curve at the selected luma input level.

[0112] Example 13 provides one or more non-transitory computer-readable media of any above example, wherein determining that the gap exists in the luminance histogram comprises identifying a pixel-free portion of the luminance histogram, wherein approximately no pixels have a luminance level in the pixel-free portion.

[0113] Example 14 provides one or more non-transitory computer-readable media of any of the above examples, further comprising generating an output image frame using the final tone mapping curve, wherein the output image frame includes a first output portion corresponding to the main portion of the input image frame, and wherein the first output portion has increased contrast compared to the main portion of the input image frame.

[0114] Example 15 provides a device comprising: a computer processor for executing computer program instructions; and a non-transitory computer-readable memory storing computer program instructions executable by the computer processor to perform operations, the operations comprising: receiving an input image frame from an imager; generating a luminance histogram; determining that there is a gap in the luminance histogram, wherein the gap is between a main portion of the input image frame having low luminance and a small portion of the input image frame having high luminance; generating a first tone mapping curve for the main portion of the input image frame, wherein the first tone mapping curve maps a measured input luminance level to a target output luminance level; identifying a selected luminance output target point on the first tone mapping curve; generating a second tone mapping curve from the selected luminance output target point to a maximum luminance level of the luminance histogram; and generating a final tone mapping curve, wherein the final tone mapping curve includes the first tone mapping curve and the second tone mapping curve from a minimum luminance input to the selected luminance output target point.

[0115] Example 16 provides the apparatus of Example 15, wherein the operation further comprises determining a percentile of the main portion, wherein the percentile represents a first number of pixels in the main portion compared to a total number of pixels in the input image frame.

[0116] Example 17 provides the apparatus of any above example, wherein the operations further comprise determining a selected luma input level corresponding to the selected luma output target point on the first tone mapping curve.

[0117] Example 18 provides the apparatus of any above example, wherein the operations further comprise determining a gradient of the first tone mapping curve at the selected luma input level.

[0118] Example 19 provides the apparatus of any above example, wherein determining the presence of the gap in the luminance histogram comprises identifying a pixel-free portion of the luminance histogram, wherein approximately no pixels have a luminance level in the pixel-free portion.

[0119] Example 20 provides an apparatus of any of the above examples, wherein the operation further comprises generating an output image frame using the final tone mapping curve, wherein the output image frame comprises a first output portion corresponding to the main portion of the input image frame, and wherein the first output portion has increased contrast compared to the main portion of the input image frame.

[0120] Example 21 provides a computer-implemented method, one or more non-transitory computer-readable media, and / or apparatus of any of the above examples, wherein determining the gradient of the first tone mapping curve at the selected luma input level comprises determining the gradient based on the selected luma output target point and an input point corresponding to the selected luma input level.

[0121] Example 22 provides a computer-implemented method, one or more non-transitory computer-readable media, and / or apparatus of any of the above examples, wherein determining the gradient of the first tone mapping curve at the selected luma input level comprises determining the gradient based on a first pair of points on the first tone mapping curve and a second pair of points on the first tone mapping curve, wherein the first pair of points comprises the selected luma output target point and an input point corresponding to the selected luma input level.

[0122] Example 23 provides the computer-implemented method, one or more non-transitory computer-readable media, and / or apparatus of any of the above examples, wherein the first tone mapping curve comprises multiple pairs of adjacent points, and wherein the second pair of points is adjacent to the first pair of points.

[0123] Example 24 provides the computer-implemented method, one or more non-transitory computer-readable media, and / or apparatus of any of the above examples, wherein the first pair of points includes the selected luma output target point and an input point corresponding to the selected luma input level.

[0124] The above description of the illustrated implementations of the present disclosure, including the content described in the abstract, is not intended to be exhaustive or to limit the present disclosure to the precise form disclosed. Although specific implementations and examples of the present disclosure are described herein for illustrative purposes, as will be appreciated by those skilled in the relevant art, various equivalent modifications are possible within the scope of the present disclosure. These modifications may be made to the present disclosure in light of the above detailed description.

Claims

1. A computer-implemented method comprising: receiving an input image frame from an imager; Generate a luminance histogram; determining that a gap exists in the luminance histogram, wherein the gap is between a main portion of the input image frame having low luminance and a small portion of the input image frame having high luminance; generating a first tone mapping curve for the primary portion of the input image frame, wherein the first tone mapping curve maps a measured input luminance level to a target output luminance level; identifying a selected luminance output target point on the first tone mapping curve; generating a second tone mapping curve from the selected luminance output target point to a maximum luminance level of the luminance histogram; as well as A final tone mapping curve is generated, the final tone mapping curve including the first tone mapping curve and the second tone mapping curve from a minimum luminance input to the selected luminance output target point.

2. The computer-implemented method of claim 1 , further comprising determining a percentile for the principal portion, wherein the percentile represents a first number of pixels in the principal portion compared to a total number of pixels in the input image frame. The computer-implemented method of claim 2 , wherein the percentile is greater than 90%.

4. The computer-implemented method of any of claims 1-3, further comprising determining a selected luma input level corresponding to the selected luma output target point on the first tone mapping curve. 5 . The computer-implemented method of claim 4 , further comprising determining a gradient of the first tone mapping curve at the selected luma input level.

6. The computer-implemented method of claim 5, wherein determining the gradient of the first tone mapping curve at the selected luma input level comprises determining the gradient based on the selected luma output target point and an input point corresponding to the selected luma input level.

7. The computer-implemented method of claim 5 , wherein determining the gradient of the first tone mapping curve at the selected luma input level comprises determining the gradient based on a first pair of points on the first tone mapping curve and a second pair of points on the first tone mapping curve, wherein the first pair of points comprises the selected luma output target point and an input point corresponding to the selected luma input level.

8. The computer-implemented method of claim 7, wherein the first tone mapping curve comprises a plurality of pairs of adjacent points, and wherein the second pair of points is adjacent to the first pair of points.

9. The computer-implemented method of claim 7, wherein the first pair of points comprises the selected luma output target point and an input point corresponding to the selected luma input level.

10. The computer-implemented method of any of claims 1-3, wherein determining that the gap exists in the luminance histogram comprises identifying a pixel-free portion of the luminance histogram, wherein approximately no pixels have a luminance level in the pixel-free portion.

11. The computer-implemented method of any of claims 1-3, further comprising generating an output image frame using the final tone mapping curve, wherein the output image frame comprises a first output portion corresponding to the primary portion of the input image frame, and wherein the first output portion has increased contrast compared to the primary portion of the input image frame.

12. One or more non-transitory computer-readable media storing instructions executable to perform operations comprising: receiving an input image frame from an imager; Generate a luminance histogram; determining that a gap exists in the luminance histogram, wherein the gap is between a main portion of the input image frame having low luminance and a small portion of the input image frame having high luminance; generating a first tone mapping curve for the primary portion of the input image frame, wherein the first tone mapping curve maps a measured input luminance level to a target output luminance level; identifying a selected luminance output target point on the first tone mapping curve; generating a second tone mapping curve from the selected luminance output target point to a maximum luminance level of the luminance histogram; as well as A final tone mapping curve is generated, the final tone mapping curve including the first tone mapping curve and the second tone mapping curve from a minimum luminance input to the selected luminance output target point.

13. The one or more non-transitory computer-readable media of claim 12, the operations further comprising determining a percentile for the principal portion, wherein the percentile represents a first number of pixels in the principal portion compared to a total number of pixels in the input image frame.

14. The one or more non-transitory computer-readable media of claim 13, wherein the percentile is greater than 90%.

15. The one or more non-transitory computer-readable media of any of claims 12-14, the operations further comprising determining a selected luma input level corresponding to the selected luma output target point on the first tone mapping curve.

16. The one or more non-transitory computer-readable media of claim 15, the operations further comprising determining a gradient of the first tone mapping curve at the selected luma input level.

17. One or more non-transitory computer-readable media according to claim 16, wherein determining the gradient of the first tone mapping curve at the selected luma input level comprises determining the gradient based on the selected luma output target point and an input point corresponding to the selected luma input level.

18. One or more non-transitory computer-readable media according to any of claims 12-14, wherein determining that the gap exists in the luminance histogram comprises identifying a pixel-free portion of the luminance histogram, wherein approximately no pixels have a luminance level in the pixel-free portion.

19. One or more non-transitory computer-readable media according to any of claims 12-14, further comprising using the final tone mapping curve to generate an output image frame, wherein the output image frame includes a first output portion corresponding to the main portion of the input image frame, and wherein the first output portion has increased contrast compared to the main portion of the input image frame.

20. An apparatus comprising: a computer processor for executing computer program instructions; as well as a non-transitory computer-readable memory storing computer program instructions executable by the computer processor to perform operations comprising: receiving an input image frame from an imager; Generate a luminance histogram; determining that a gap exists in the luminance histogram, wherein the gap is between a main portion of the input image frame having low luminance and a small portion of the input image frame having high luminance; generating a first tone mapping curve for the primary portion of the input image frame, wherein the first tone mapping curve maps a measured input luminance level to a target output luminance level; identifying a selected luminance output target point on the first tone mapping curve; generating a second tone mapping curve from the selected luminance output target point to a maximum luminance level of the luminance histogram; and A final tone mapping curve is generated, the final tone mapping curve including the first tone mapping curve and the second tone mapping curve from a minimum luminance input to the selected luminance output target point.

21. The apparatus of claim 20, wherein the operations further comprise determining a percentile for the principal portion, wherein the percentile represents a first number of pixels in the principal portion compared to a total number of pixels in the input image frame.

22. The apparatus of any of claims 20-21, wherein the operations further comprise determining a selected luma input level corresponding to the selected luma output target point on the first tone mapping curve.

23. The apparatus of claim 22, wherein the operations further comprise determining a gradient of the first tone mapping curve at the selected luma input level.

24. The apparatus of any of claims 20-21, wherein determining that the gap exists in the luminance histogram comprises identifying a pixel-free portion of the luminance histogram, wherein approximately no pixels have a luminance level in the pixel-free portion.

25. An apparatus according to any of claims 20-21, wherein the operation further comprises generating an output image frame using the final tone mapping curve, wherein the output image frame comprises a first output portion corresponding to the main portion of the input image frame, and wherein the first output portion has increased contrast compared to the main portion of the input image frame.

Images