Method and apparatus for reducing dynamic range of an image

Abstract

Methods and apparatus are provided for reducing the dynamic range of an image. A first image and a second image have pixel values of a first bit depth. One or more peak regions and a representative pixel value for each peak region are identified in a histogram of pixel values in the first image. For pixel values in the first image that are within a peak region, a pixel value difference is computed as the difference between the pixel value of the peak region and the representative pixel value. For pixel values in the first image that are outside of a peak region, the pixel value difference is zero. Pixel values of the second image are transformed to reduce the bit depth to a second bit depth. The pixel value differences computed from the first image are then added to the pixel values of the transformed second image.

Description

Technical Field

[0001] This application relates to the field of image processing. Specifically, it relates to a method and apparatus for reducing the dynamic range of an image. Background Technology

[0002] The dynamic range of an image captured by an image sensor is related to the bit depth (i.e., the number of bits used to represent the pixel value in each pixel of the image). Higher bit depths result in images with higher dynamic ranges. High dynamic range image sensors have entered the market in recent years. For example, for thermal imagers, there are high dynamic range calorimeter sensors that provide images where each pixel value is represented by up to 21 bits. The reason for using such a high dynamic range sensor is its high sensitivity to small temperature changes, while also being sensitive to temperature over a wide temperature range.

[0003] In some cases, such as due to technical limitations of other components of the imaging system, it is necessary to reduce the dynamic range of the image generated by the image sensor. One such case, described in patent document US2008 / 0019608A1, is when the output of an image sensor with a high dynamic range is to be presented on a display device with a lower dynamic range than that of the image sensor. Another case is when an image generated by a high dynamic range image sensor is to be processed in an image processing pipeline designed to process images with a lower dynamic range than that provided by the image sensor. For example, the image processing pipeline may be designed to process images with a 10-bit bit depth, while the image sensor provides an image with a 21-bit bit depth.

[0004] Various methods exist for reducing the dynamic range of an image from a first-order depth to a lower second-order depth. These include applying linear transformations to the pixel values ​​in the image to map them from a range of pixel values ​​represented in the first-order depth to a range that can be represented in the lower second-order depth. The advantage of linear transformations is their computational inexpensive implementation, but they have the disadvantage of losing much detail in the image when the transformation is applied. More complex methods also exist that apply nonlinear transformations, such as histogram equalization, to the pixel values. These more complex methods are generally more computationally expensive, but they have the advantage of preserving more detail in the image.

[0005] However, there are situations where there are no computational resources to implement more complex methods, but it is still necessary to preserve details in the image. One such example is when a surveillance camera with limited processing power needs to detect and identify objects in image data, a process that demands a high level of detail in the image. Therefore, there is room for improvement. Summary of the Invention

[0006] In view of the above, one object of the present invention is to alleviate the above problems and to provide a method for reducing the dynamic range of an image, which can be implemented in a computationally efficient manner while preserving the details in the image.

[0007] According to the first aspect, the above objective is achieved by a method for reducing the dynamic range of an image as defined by the first independent claim. The method includes:

[0008] The system receives a first image and a second image, each having a pixel value representing a first depth, wherein each pixel in the first image has a spatially corresponding pixel in the second image, and wherein the first image and the second image are either the same image in a video sequence or consecutive images in a video sequence.

[0009] Analyze the histogram of pixel values ​​in the first image to identify one or more peak regions and representative pixel values ​​for each peak region.

[0010] Calculate the pixel value difference for each pixel in the first image, where,

[0011] For each pixel in the first image that has a pixel value within the peak region, the pixel value difference is the difference between the pixel value of the pixel in the first image and the representative pixel value of the peak region, and

[0012] For each pixel in the first image that has a pixel value outside one or more peak regions, the pixel value difference is zero.

[0013] The second image is transformed by applying a transform that reduces the bit depth of each pixel value in the second image from a first bit depth to a lower second bit depth, thereby reducing its dynamic range.

[0014] For each pixel in the transformed second image, the calculated pixel value difference of the corresponding pixel in the first image is added to the pixel value of the transformed second image.

[0015] Using this method, details in the form of calculated pixel value differences are added to the second image after it has undergone a bit-depth reduction transformation. This compensates for the loss of detail during the bit-depth reduction transformation. As a result, computationally efficient transformations, such as the previously mentioned linear transformations, can be used, even if they result in the loss of detail during the transformation.

[0016] The pixel value difference is calculated from a histogram of pixel values ​​in a first image with a higher dynamic range (i.e., a first image with a first-order depth). When calculating the pixel value difference, only pixels in the first image with pixel values ​​that fall within the peak regions of the histogram are assigned a non-zero pixel value difference. In this way, noise can be suppressed to a high degree, as noise tends to be distributed uniformly in the histogram rather than forming peaks. The non-zero pixel value difference is calculated as the difference between the pixel value of a pixel and the representative pixel value of the peak region. As a result of this method, the pixel value difference tends to preserve the shape of the peaks in the histogram. Furthermore, as will be described below, by calculating the difference in this way, it is found that although rescaling may be necessary in some cases, the difference generally has a size that allows it to be represented using a lower second-order depth.

[0017] Therefore, it should be noted that the present invention is particularly advantageous when the pixel values ​​in an image cover a wide range, but where the interest lies in distinguishing details (i.e., small differences in pixel values) in several different parts of that range. One such case is in thermal images depicting scenes where temperatures span hundreds of degrees Celsius, but where the interest lies in distinguishing details at both the lower end of that range (e.g., corresponding to human temperature) and the upper end (e.g., corresponding to the temperature of fire). If linear stretching is applied in such cases without enhancing the details as suggested by the present invention, much detail will be lost, which in the worst case would render the resulting image useless for its intended purpose.

[0018] The dynamic range of an image can generally be defined as the ratio between the maximum and minimum pixel values ​​in the image.

[0019] As mentioned, the dynamic range of an image is related to its bit depth. Bit depth is the number of bits used to represent the pixel value in each pixel of an image. A higher bit depth results in an image with a higher dynamic range compared to one with a lower bit depth.

[0020] In the context of this application, reducing the dynamic range of an image refers to reducing the bit depth of the image.

[0021] The first and second images mentioned in this article can be the same image. In that case, it should be understood that receiving that image in the receiving step of the method is sufficient. However, it is also conceivable that they are implementations of consecutive images in a video sequence. For example, the second image could be the next image after the first image in the video sequence. In either case, it should be understood that the first and second images typically have the same pixel size, such as both having an MxN pixel size.

[0022] The first and second images have spatially corresponding pixels. Specifically, each pixel in the first image has a spatially corresponding pixel in the second image, and vice versa. Pixels in the first and second images are spatially corresponding if they have the same pixel coordinates. A pixel in the first image with pixel coordinates (i,j) spatially corresponds to a pixel in the second image with pixel coordinates (i,j).

[0023] Pixel value can also be referred to as pixel intensity. For thermal images, pixel value or pixel intensity reflects the amount of IR radiation from the scene and is proportional to temperature. For visible light images, pixel value or pixel intensity reflects the amount of radiation in the visible portion of the spectrum, also known as luminance.

[0024] In cases where there is a relatively large difference between the first and second bit depths, the pixel value difference may have a size that cannot be represented at the scale provided by the lower second bit depth. In these cases, it may be advantageous to rescale the pixel value difference before adding it to the pixel values ​​in the transformed second image. Therefore, the method may further include scaling the pixel value difference by a multiplication factor for each pixel in the transformed second image before adding the calculated pixel value difference of the spatially corresponding pixels in the first image to the pixel values ​​in the transformed second image.

[0025] For all pixel value differences, the multiplication factor can be the same. For example, it can be a predetermined factor that depends on the first and second bit depths. Alternatively, each peak region can be associated with a multiplication factor, where the pixel value difference calculated from pixels in the first image that have pixel values ​​within the peak region is scaled with the multiplication factor associated with that peak region. In this way, for example, the multiplication factor can be adjusted to account for different peak widths. For example, wider peaks generally result in larger pixel value differences than narrower peaks. Thus, a smaller multiplication factor may be needed for wider peaks than for narrower peaks in order to accommodate the pixel value differences within a range of values ​​that can be represented at a lower second bit depth. Alternatively, the multiplication factor can be set to assign different importance to different peaks. For example, one peak in the histogram may be considered more important than another and therefore associated with a larger multiplication factor. For example, in the case of a thermal imager, a peak in a particular temperature range (corresponding to humans or a fire that may be starting) may be considered more important than peaks in other temperature ranges.

[0026] When pixel value differences are added to the pixel values ​​of the transformed second image, there is a risk that the resulting sum will be greater than the maximum value that can be represented in the second bit depth or less than the minimum value that can be represented in the second bit depth. If this occurs, and no countermeasures are taken, the values ​​may "loop around," which will lead to unpleasant artifacts. Therefore, the method may further include, after the addition step, limiting the pixel values ​​in the transformed second image to between the minimum and maximum values ​​that can be represented in the second bit depth. This process is sometimes referred to as clamping.

[0027] As used herein, the peak region of a histogram refers to the interval of pixel values ​​surrounding a peak in the histogram. Pixel values ​​within this interval may be referred to as falling within or belonging to the peak region. The histogram specifies the frequency values ​​of pixel values ​​used in the first image. The frequency value specifies how often different pixel values ​​appear in the first image, i.e., how many times. The frequency value may be further normalized by the number of pixels in the image. According to an implementation, the peak region of the histogram is identified as an interval of at least a predetermined number of consecutive pixel values ​​whose frequency values ​​exceed a predetermined frequency threshold. In other words, for a consecutive interval of pixel values ​​of at least a certain length, the frequency values ​​should remain above the predetermined frequency threshold so that the interval qualifies as a pixel region. In other words, two thresholds are involved in the identification: a first threshold, a frequency threshold, which specifies the minimum frequency of pixel values ​​in the peak region; and a second threshold, a peak width threshold, which specifies the minimum width of the peak region.

[0028] The representative pixel value of a peak region is the representative pixel value among the pixel values ​​falling within the peak region. The representative pixel value can be a statistic describing the central trend of pixel values ​​within the peak region, i.e., the "middle" or "center," or the location of the peak region. For example, the representative pixel value of a histogram's peak region can be one of the following: the center of the interval defining the lowest and highest pixel values ​​of the peak region of the histogram; the statistical average of the pixel values ​​falling within the peak region of the histogram in the first image; the mode of the peak region of the histogram (i.e., the most frequent pixel value in the peak region); and the median of the pixel values ​​falling within the peak region of the histogram in the first image.

[0029] As described above, the underlying implementation of this invention is that because the pixel value difference calculated from the higher dynamic range image is added to the second image after the second image is transformed into a lower dynamic range image, it doesn't matter much if detail is lost during the transformation. This makes it possible to use computationally efficient "simple" transformations, even if it results in a loss of detail during the transformation. For example, a transformation that reduces the bit depth of each pixel value in the second image can include linear stretching. Linear stretching is a linear mapping from a range of pixel values ​​that can be represented by a first bit depth to a range of pixel values ​​that can be represented by a second bit depth.

[0030] The method may further include buffering the calculated pixel value differences in an image buffer. In this way, the pixel value differences can be temporarily stored before being added to the transformed second image.

[0031] The first and second images can generally be any type of image, such as those captured by a camera sensitive to light in the visible portion of the spectrum. However, in a particularly advantageous embodiment, the first and second images are thermal images. The use of high dynamic range sensors, such as those providing images with a bit depth of 18 to 21 bits, is generally more relevant to thermal imaging than to visible light imaging. This is because thermal sensors typically have lower resolution than visible light sensors, and therefore can afford the additional processing costs associated with processing images with higher bit depths. For visible light sensors with higher resolution, the processing costs would be infeasible at such high bit depths. Another reason the present invention is particularly advantageous to thermal images is that it is generally desirable for thermal images to include as much detail as possible. Therefore, the ability to reduce the dynamic range of an image while still enhancing detail is especially important for thermal images. This contrasts with visible light images, whose primary purpose is "aesthetic appeal" rather than conveying detail.

[0032] According to a second aspect of the invention, the above-mentioned objective is achieved by a processing apparatus for reducing the dynamic range of an image, as defined by the second independent claim.

[0033] According to a third aspect of the invention, the above-mentioned objective is achieved by a non-transitory computer-readable medium as defined by the third independent claim.

[0034] The second and third aspects may generally have the same features and advantages as the first aspect. It should be further noted that the invention relates to all possible combinations of features, unless otherwise expressly stated. Attached Figure Description

[0035] The above and other objects, features, and advantages of the invention will be better understood by referring to the following illustrative and non-limiting detailed description of embodiments of the invention, wherein like reference numerals will be used for similar elements, wherein:

[0036] Figure 1 The illustration shows a system in which example implementation methods can be carried out.

[0037] Figure 2 The schematic illustration shows two different images being input to the processing apparatus of the device according to an exemplary embodiment.

[0038] Figure 3 The schematic illustration shows a processing device according to an embodiment where a single image is input to the device.

[0039] Figure 4 This is a flowchart of a method for reducing the dynamic range of an image.

[0040] Figure 5 Schematic diagram Figure 4 The various steps of the method.

[0041] Figures 6a to 6d The diagram shows that Figure 4 The result when the method is applied to an image. Detailed Implementation

[0042] The invention will now be described more fully with reference to the accompanying drawings, in which embodiments of the invention are illustrated.

[0043] Figure 1 The illustration shows a system 100 in which embodiments of the present invention can be implemented. System 100 includes an image sensor 102, a processing device 104, and an image processing pipeline 106. System 100 may be included in a camera such as a thermal imager or a visible light camera.

[0044] Image sensor 102 captures images with pixel values ​​having a first depth k. This means that k bits are used to represent each pixel value, and correspondingly, there are 2^k bits for each pixel. k There are several possible pixel values. The precise value of the first depth can vary depending on which image sensor is used, but in one embodiment, the first depth can be equal to 16.

[0045] The wavelength to which the image sensor is sensitive can vary between implementations. For example, image sensor 102 may be sensitive in the infrared portion of the spectrum, for example, it may be a calorimeter sensor, or it may be sensitive in the visible portion of the spectrum. Image sensor 102 may be further arranged to capture a sequence of images constituting image frames of a video sequence.

[0046] Image processing pipeline 106 processes images captured by image sensor 102. However, image processing pipeline 106 is only capable of processing images with a second depth kj lower than the first depth k. Accordingly, image processing pipeline 106 is only able to process images with at most 2 kj per pixel. k-j An image with possible values. As is known in the art, the image processing pipeline 106 includes a number of sequential processing steps, such as for enhancing and compressing image data. Examples of such processing steps include noise reduction, global / local detail enhancement (such as CLAHE processing), sharpening, etc. For the purposes of this disclosure, a standard image processing pipeline 106 may be used.

[0047] In order for the image processing pipeline 106 to process the image captured by the image sensor 102, the bit depth of the image first needs to be reduced from the first bit depth k to the second bit depth kj. As will be explained, such reduction is performed by the processing device 104.

[0048] exist Figure 2 and Figure 3 The processing device 104 is illustrated in more detail below. The processing device 104 includes a receiver 202, a bit depth reducer 204, a histogram analyzer 206, a pixel value difference calculator 208, and an adder 210.

[0049] The processing device 104 therefore includes various components 202, 204, 206, 208, 210 configured to implement the functions of the device 104. In particular, each illustrated component corresponds to a function of the device 104. Generally, the device 104 may include circuitry configured to implement the components 202, 204, 206, 208, 210, and more specifically, to implement their functions.

[0050] In hardware implementation, each of components 202, 204, 206, and 208 may correspond to a dedicated circuit specifically designed to provide the functionality of the component. The circuit may be in the form of one or more application-specific integrated circuits (ASICs) or one or more field-programmable gate arrays (FPGAs). For example, bit depth reducer 204 may therefore include circuitry for transforming an image to reduce its dynamic range during use.

[0051] In a software implementation, the circuitry may be replaced by a processor, such as a central processing unit or a graphics processing unit, associated with computer code instructions stored on a (non-transitory) computer-readable medium, such as non-volatile memory, that cause device 104 to perform any of the methods disclosed herein. Examples of non-volatile memory include read-only memory, flash memory, ferroelectric RAM, magnetic computer storage devices, and optical discs. In the software implementation, components 202, 204, 206, and 208 may therefore each correspond to a portion of computer code instructions stored on a computer-readable medium that, when executed by a processor, causes device 104 to perform the function of the component.

[0052] It should be understood that a combination of hardware and software implementation is also possible, meaning that the functionality of some components in components 202, 204, 206, 208, and 210 is implemented in hardware, while other components are implemented in software.

[0053] Now refer to Figures 1 to 3 , Figure 5 as well as Figure 4 The flowchart is used to explain the operation of the processing device 104 when performing a method for reducing the dynamic range of an image.

[0054] In step S02, receiver 202 receives first image 221 and second image 222. First image 221 and second image 222 are also... Figure 5 As shown in the diagram. The first image 221 and the second image 222 typically have the same size, such as M x N pixels, and they have spatially corresponding pixels. That is, for each pixel position 506, 508 in the first image 221, there exists a spatially corresponding pixel position 506, 508 in the second image 222, and vice versa. The pixel values ​​in the first image 221 and the second image 222 have a first depth k. As explained, this means that there exist integer values ​​such as 0 to 2. k -1 of 2 k Several possible pixel values. The first image 221 and the second image 222 are received from the image sensor 102. In some embodiments, and as... Figure 2 As illustrated, the first image 221 and the second image 222 are consecutive images of a video sequence 220 captured by the image sensor 102. For example, in the video sequence 220, the first image 221 may be an image frame immediately preceding image 222. In other embodiments, and as shown... Figure 3 As illustrated, the first image 221 and the second image 222 are the same images captured by the image sensor 102. For example, they could be the same images from video sequence 220. In the latter embodiment, it should be understood that it is sufficient for the receiver 202 to receive only one image.

[0055] The first image 221 is forwarded to the histogram analyzer 206, which continues to analyze the histogram of pixel values ​​in the first image 221 in step S04. This... Figure 5 Further illustrations are provided. Figure 5 The histogram 501 of the first image 221, calculated by histogram analyzer 206, is illustrated schematically. Histogram 501 presents the pixel values ​​in the first image 221 along the horizontal axis. Although full-scale is not used for the pixel values ​​in this example, it is evident on the scale that pixel values ​​can take values ​​from 0 to 2 in this case. k Values ​​between -1. On the vertical axis, histogram 501 presents the frequency values ​​of pixel values ​​in the first image 221, that is, the number of pixels in the first image 221 with a particular pixel value. As is known in the art, grouping of pixel values ​​can be used when calculating a histogram. In that case, pixel values ​​are grouped into groups or intervals of equal width, and the frequency value of the group corresponds to the number of pixel values ​​within the group. Further, as is known in the art, the histogram can be normalized by the total number of pixels in the first image 221.

[0056] Analysis of histogram 501 involves identifying one or more peak regions within the histogram. A peak region is a range of pixel values ​​surrounding a peak in the histogram. Figure 5 In the embodiments, three peak regions 502-1, 502-2, and 502-3 were identified. To find peak regions 502-1, 502-2, and 502-3, histogram analyzer 206 can apply a predetermined frequency threshold T1 to the frequency values ​​of histogram 501. Peak regions can then be identified as intervals of pixel values ​​whose frequency values ​​exceed the threshold T1. In some embodiments, the interval to which histogram analyzer 206 can apply pixel values ​​should further include the condition of having at least a predetermined number of consecutive pixel values ​​with frequency values ​​exceeding the frequency threshold T1. That is, the interval should have a width of at least a peak width threshold T2. In this way, peak regions consisting of only a single or a few pixel values ​​can be avoided.

[0057] The peak width threshold T2 is a design parameter that can be set, for example, as a specific percentage of the total number of pixel values ​​that can be represented in the first depth. For instance, it could be set to 5% of the total number of pixel values ​​that can be represented in the first depth. The peak width threshold T2 can also be set based on the noise of sensor 102. For example, for a thermal sensor (radiometric calorimeter sensor), the noise equivalent temperature difference (NETD) is a measure of how well the sensor can distinguish small differences between thermal radiations in an image. The peak width threshold T2 can be set equal to NETD. Note that NETD can depend on both the ambient temperature and the temperature of the target (i.e., the pixel values ​​in the image). This means that the peak width threshold T2 can depend on the pixel values ​​in the candidate peak region so that it takes a different value for the relatively lower portion of the range of possible pixel values ​​compared to the relatively higher portion of the range of possible pixel values. For example, the peak width threshold T2 for the temperature of a human object in a thermal image will be different from the peak width threshold T2 for a hot object at several hundred degrees Celsius.

[0058] Similarly, the value of the frequency threshold T1 is a design parameter. The frequency threshold T1 can be set depending on the range of pixel values ​​represented in the first image 221, that is, the range spanned by the lowest and highest pixel values ​​in the first image 221. The wider the range, the lower the frequency threshold T1. For example, consider, on the one hand, a room temperature scene of a human object depicted by a thermal sensor. In such a scene, pixel values ​​would be in the range of, for example, 18-34 degrees Celsius, and most pixel values ​​within this range would likely be represented in the image. On the other hand, consider a scene with a cold sky, moderate outdoor temperature, a human object, and a very hot machine. In such a scene, pixel values ​​would be in the range of 0-200 degrees Celsius, but many pixel values ​​within this range would not be represented. The former of these two scenarios would require a higher frequency threshold T1 than the latter. Note that because the frequency threshold T1 can depend on the scene content, that is, the range of pixel values ​​represented in the first image 221, it can vary between frames of the video sequence.

[0059] In step S04, the histogram analyzer further identifies representative pixel values ​​for each peak region 502-1, 502-2, and 502-3. Figure 5 In the process, representative pixel values ​​V1, V2, and V3 of peak regions 502-1, 502-2, and 502-3 are identified respectively.

[0060] Histogram analyzer 206 can calculate representative pixel values ​​V1, V2, and V3 for peak regions 502-1, 502-2, and 502-3 from the pixel values ​​falling within peak regions 502-1, 502-2, and 502-3. This can be performed in different ways.

[0061] In one implementation, the representative pixel values ​​V1, V2, and V3 of peak regions 502-1, 502-2, and 502-3 are calculated as the center of the interval defining the lowest and highest pixel values ​​of the peak regions 502-1, 502-2, and 503-3 of histogram 501. Figure 5 This is the case in the example. More specifically, as mentioned above, the peak region is the interval of pixel values ​​surrounding the peak in the histogram. This interval has a lowest pixel value and a highest pixel value, and the interval extends between these two pixel values. The representative pixel value can be calculated as the center of the interval, that is, calculated as the average of the lowest and highest pixel values ​​in the interval. This way of calculating the representative pixel value is computationally efficient and provides particularly good results for peaks with symmetrical shapes, such as peaks that follow a Gaussian distribution.

[0062] In another embodiment, representative pixel values ​​V1, V2, and V3 of peak regions 502-1, 502-2, and 502-3 are calculated as the statistical average of pixel values ​​in the first image 221 that fall within peak regions 502-1, 502-2, and 502-3 of histogram 501. When calculating the statistical average, the distribution of pixel values ​​in the peak regions, i.e., their frequency values, is also considered in the calculation. More specifically, each pixel value in the peak region is weighted proportionally to its frequency value. This embodiment has a slightly higher computational cost, but provides good results when the peaks have a skewed shape.

[0063] In another embodiment, the representative pixel values ​​V1, V2, and V3 of the peak regions 502-1, 502-2, and 502-3 are calculated as the mode of the peak regions 502-1, 502-2, and 503-3 of the histogram 501, that is, as the most frequent pixel values ​​within the peak regions.

[0064] In yet another embodiment, the representative pixel values ​​V1, V2, V3 of the peak regions 502-1, 502-2, 502-3 are calculated as the median of the pixel values ​​in the peak regions of the histogram in the first image 221.

[0065] In step S06, the pixel value difference calculator 208 then continues to calculate the pixel value difference for each pixel in the first image 221. The calculated pixel value differences may be temporarily stored in an image buffer 504, which stores the pixel value difference for each pixel in the first image 221. As will be described later, the pixel value differences in buffer 504 will act as a filter to enhance details in a scaled-down version of the second image 222.

[0066] The pixel value difference is calculated differently depending on whether the pixel value falls within the peak regions 502-1, 502-2, and 502-3. Therefore, to calculate the pixel value difference in the first image 221, the pixel value difference calculator 208 first checks whether the pixel value falls within the peak regions 502-1, 502-2, and 502-3. (Reference) Figure 5 In one embodiment, it is assumed that pixel position 506 of the first image 221 has a pixel value P1 within peak region 502-3, and pixel position 508 has a pixel value P2 that is not within any peak region 502-1, 502-2, 502-3.

[0067] For a pixel in the first image 221 that has a pixel value within the peak region, the pixel value difference is calculated as the difference between that pixel value and the representative pixel value of the peak region. For example, the pixel value difference at pixel position 506 in the first image 221 is therefore calculated as D1 = P1 - V3. Note that the pixel value difference D1 will be positive when the pixel value P1 is greater than the representative pixel value V3, and negative if the pixel value P1 is lower than the representative pixel value V3. The pixel value difference D1 can be stored in the corresponding pixel position 506 of the image buffer 504.

[0068] For pixels in the first image 221 that have pixel values ​​outside one or more peak regions, the pixel value difference is instead set to zero. Figure 5 In this embodiment, the pixel value difference at pixel position 508 in the first image 221 will therefore be set to zero. The value of zero can be stored in the corresponding pixel position 508 of the image buffer 504.

[0069] When this step is performed on each pixel in the first image 221, the image buffer 504 will fill the pixel value difference of each pixel in the first image 221. Depending on whether the pixel value of the corresponding pixel position in the first image 221 is within or outside the peak regions 502-1, 502-2, 502-3, some of these differences will be non-zero, while others will be zero.

[0070] In step S08, the bit depth reducer 204 transforms the second image 222, and specifically transforms its pixel values ​​to reduce the dynamic range. As a result of the transformation, a transformed second image 512 is generated. Note that the transformed second image 512 and the second image 222 will have the same size and spatially corresponding pixels. However, they will have different pixel values.

[0071] To transform the second image 222, the bit depth reducer 204 applies a transformation to the pixel values ​​of the second image, reducing the bit depth from a first bit depth k to a lower second bit depth kj, where j ≥ 1. The first bit depth can represent a range [0, 2]. k Integer pixel values ​​in [-1], while the range that can be represented by the second bit depth is [0, 2]. k-j Integer pixel values ​​in [-1]. The transformation can therefore be viewed as a function or mapping. That is, the range [0, 2] k Integer pixel values ​​in [-1] are mapped to the range [0, 2]. k-j A function of integer pixel values ​​in [-1]. In box 510, Figure 5 The diagram illustrates a histogram 503 of the pixel values ​​in the second image 222 before applying the transformation f and a histogram 505 after applying the transformation f.

[0072] Generally, any known bit-depth reduction transform can be used in step S08. In particular, computationally efficient transforms can be used while still achieving acceptable final results. In one implementation, the transform is a so-called linear stretching of the pixel value. Linear stretching may include two sub-steps: first, applying a gain to reduce the pixel value represented by the first bit depth, and then offsetting the resulting value. In one embodiment, the gain G can be calculated according to the following relationship:

[0073]

[0074] Where R2 is the number of possible pixel values ​​at the second bit depth, i.e., R2 = 2. k-j R1 is the difference between the maximum and minimum pixel values ​​at the first depth in the second image 222. The offset O can be calculated according to the following formula:

[0075] O = 0.5R² - G·M,

[0076] Where M is the average pixel value in the second image 222 represented in the second bit depth. By applying such an offset, the average pixel value in the second image 222 will be mapped to the center of the range of pixel values ​​that can be represented in the second bit depth.

[0077] Step S08 is typically performed after steps S04 and S06. When the first image 221 and the second image 222 are as follows... Figure 3 This is especially true when the images are identical, as illustrated in the diagram. In that case, the images can be processed first to calculate the pixel value differences. Once these values ​​have been calculated, the images are transformed to a lower bit depth. In this way, it is not necessary to store and process separate copies of the images as in parallel execution steps. When the first image 221 and the second image 222 are as shown in the diagram... Figure 2 When dealing with different images as illustrated in the diagram, the order of these steps is not important, and they can even be performed in parallel.

[0078] In step S10, adder 210 continues to add the pixel value difference to the pixel values ​​in the transformed second image 512. The resulting image is... Figure 5The image is denoted by 514. This addition is performed pixel-wise and can be accomplished by performing a pixel-wise addition of the transformed second image 512 and the image stored in the image buffer 504. As a result, for each pixel in the transformed second image 512, the calculated pixel value difference of the spatially corresponding pixel in the first image 221 is added to the pixel value in the transformed second image 512. For example, the pixel value at pixel position 506 in the transformed second image 512 is added to the pixel value difference D1 at the same pixel position 506 in the image buffer 504. In this way, the pixel value difference in the image buffer 504 can be referred to as acting as a filter for enhancing the details in the transformed second image 512.

[0079] In some implementations, the pixel value difference stored in image buffer 504 can be scaled by a multiplication factor before being added to the pixel values ​​of the transformed second image 512. The multiplication factor can be set, for example, such that the scaled maximum pixel value difference is equal to a specific percentage of the number of possible pixel values ​​that can be represented in a second bit depth, i.e., 2. k-j A specific percentage. For example, the percentage could be set to 10%. One reason for applying the multiplication factor is to reduce the risk that the sum of the pixel values ​​and pixel differences in the transformed second image 512 falls outside the range of pixel values ​​that can be represented in a second-bit depth. If this happens, the resulting pixel values ​​could end up being wrapped around pixel values ​​within that range, resulting in artifacts. Therefore, the multiplication factor can be less than 1, meaning the pixel value differences are reduced.

[0080] The same multiplication factor can be used for all pixel value differences. However, in some implementations, different peak regions can be associated with different multiplication factors. The multiplication factor can be related to the width of the peak region, that is, the width of the interval of pixel values ​​that constitutes the peak region. For example, a wider peak region can be associated with a smaller multiplication factor compared to a narrower peak region. This can be achieved, for example, by setting a multiplication factor for the peak region such that the maximum pixel value difference of the scaled peak region is equal to a specific percentage (i.e., 2) of the number of possible pixel values ​​that can be represented in a second-bit depth. k-j This can be achieved by using a specific percentage. For example, the percentage could be set to 10%. In this way, pixel values ​​in a wider peak region will be reduced more (if the multiplication factor is less than 1) or increased less (if the multiplication factor is greater than 1) compared to pixel values ​​in a narrower peak region. Another option is to correlate the multiplication factor with the importance of the peak region, so that a higher multiplication factor is applied to more important peak regions rather than less important ones. For example, again referring to thermal images, peak regions corresponding to the temperature range of humans can be given higher importance than peak regions corresponding to the temperature range of cold skies.

[0081] After the pixel value difference in image buffer 504 has been added to the pixel values ​​of the transformed second image 512, the pixel values ​​in the resulting image 514 can be constrained such that they are between a minimum and a maximum value that can be represented in a second bit depth. In this case, the pixel values ​​in the resulting image 514 are therefore constrained to be at most equal to 2. k-j -1, and not less than zero. Accordingly, if the sum of the pixel value difference and the pixel value in the transformed second image 512 would result in a value greater than the maximum value, then it will be set to equal the maximum value. Similarly, if the sum of the pixel value difference and the pixel value in the transformed second image 512 would result in a value less than the minimum value, then it will be set to equal the minimum value.

[0082] Figures 6a to 6d The diagram illustrates how the above method is used to reduce Figure 6b An embodiment of the dynamic range of image 621 illustrated in the figure. In this embodiment, the first image and the second image mentioned above are the same image, namely image 621. Image 621 has a first-bit depth of 16 bits, which means 2 16 =65536 distinct pixel values ​​can be represented. When this method is applied to image 621, it reduces the bit depth to a second bit depth of 12 bits, which allows 2 12 =4096 distinct pixel values ​​are represented. The histogram 602 of image 216 is as follows: Figure 6a As shown in the diagram. It can be seen that image 621 does not include pixel values ​​above the entire range of possible pixel values, but rather includes its sub-range. A magnification of histogram 602 within this sub-range is shown below. Figure 6a Histogram 604 is shown in the image. Ten peak regions are identified from histogram 602, and a representative pixel value for each peak region is determined. In this case, the representative pixel value of the peak region is calculated as the center of the interval defining the lowest and highest pixel values ​​of the peak region. The peak regions and representative pixel values ​​are then further used to calculate the pixel value difference per pixel as described above. The original image 621 is then transformed by applying linear stretching of the pixel values. More specifically, the pixel values ​​in image 621 are linearly stretched from a first depth (16 bits) to a second depth (12 bits). In this way, the image is obtained... Figure 6c The transformed image 612 is shown in the figure. Then, the calculated pixel value difference is added to the transformed image 612 to obtain... Figure 6d Image 614 is shown in the figure. It can be seen that the details in image 614 are enhanced compared to the linearly stretched image 612.

[0083] Those skilled in the art will recognize that the above embodiments can be modified in many ways while still utilizing the advantages of the invention as shown in the above embodiments. Therefore, the invention should not be limited to the illustrated embodiments, but should be defined only by the appended claims. Furthermore, as those skilled in the art will understand, the illustrated embodiments can be combined.

Claims

1. A method for reducing the dynamic range of an image, comprising: receiving a first image and a second image of pixel values having a first bit depth, wherein each pixel in the first image has a spatially corresponding pixel in the second image, and wherein the first and second images are the same image in a video sequence or are consecutive images in a video sequence, analyzing a histogram of the pixel values in the first image to identify one or more peak regions in the histogram and a representative pixel value for each peak region, computing a pixel value difference for each pixel in the first image, wherein, for each pixel in the first image having a pixel value within a peak region, the pixel value difference is the difference between the pixel value of the pixel in the first image and the representative pixel value of the peak region, and for each pixel in the first image having a pixel value outside the one or more peak regions, the pixel value difference is zero, transforming the second image by applying a transformation that reduces the bit depth of each pixel value in the second image from the first bit depth to a lower second bit depth to reduce its dynamic range, and for each pixel in the transformed second image, adding the computed pixel value difference of the spatially corresponding pixel in the first image to the pixel value of the pixel in the transformed second image.

2. The method of claim 1, further comprising: for each pixel in the transformed second image, before adding the computed pixel value difference of the spatially corresponding pixel in the first image to the pixel value of the pixel in the transformed second image, scaling the pixel value difference by a multiplication factor.

3. The method of claim 2, wherein, The multiplication factor is the same for all pixel value differences.

4. The method of claim 2, wherein, Each peak region is associated with a multiplication factor, and wherein the pixel value difference computed from a pixel in the first image having a pixel value within a peak region is scaled by the multiplication factor associated with that peak region.

5. The method of claim 1, further comprising: after the step of adding, limiting the pixel values in the transformed second image between a minimum value and a maximum value that can be represented with the second bit depth.

6. The method of claim 1, wherein, A peak region of the histogram is an interval of pixel values around a peak in the histogram.

7. The method of claim 1, wherein, The histogram specifies frequency values of the pixel values in the first image, and wherein a peak region of the histogram is identified as an interval of consecutive pixel values for which the frequency values exceed a predetermined frequency threshold by at least a predetermined amount.

8. The method of claim 1, wherein, The representative pixel value of a peak region of the histogram is one of: a center of an interval defining a lowest pixel value and a highest pixel value of the peak region of the histogram, a statistical mean of the pixel values in the first image falling within the peak region of the histogram, a mode of the peak region of the histogram, and a median of the pixel values in the first image falling within the peak region of the histogram.

9. The method of claim 1, wherein, The transformation that reduces the bit depth of each pixel value in the second image comprises a linear stretch.

10. The method of claim 1, further comprising: buffering the computed pixel value differences in an image buffer.

11. The method of claim 1, wherein, The first and second images are thermal images.

12. A processing apparatus for reducing a dynamic range of an image, the processing apparatus comprising: circuitry configured to receive a first image and a second image having pixel values of a first bit depth, wherein each pixel in the first image has a spatially corresponding pixel in the second image, and wherein the first and second images are the same image in a video sequence, or are consecutive images in a video sequence, circuitry configured to analyze a histogram of the pixel values in the first image to identify one or more peak regions and a representative pixel value for each peak region in the histogram, circuitry configured to compute a pixel value difference for each pixel in the first image, wherein, for each pixel in the first image having a pixel value within a peak region, the pixel value difference is a difference between the pixel value of the pixel in the first image and the representative pixel value of the peak region, and for each pixel in the first image having a pixel value outside of the one or more peak regions, the pixel value difference is zero, circuitry configured to transform the second image to reduce its dynamic range by applying a transformation that reduces a bit depth of each pixel value in the second image from the first bit depth to a lower second bit depth, and circuitry configured to add, for each pixel in the transformed second image, the computed pixel value difference of the spatially corresponding pixel in the first image to the pixel value of the pixel in the transformed second image.

13. A non-transitory computer readable medium having stored thereon computer instructions that, when executed by an apparatus having processing capability, cause the apparatus to perform the method of any one of claims 1 to 11.

Images