A tone mapping apparatus and method for a high dynamic range image

Abstract

The application discloses a tone mapping device and method of a high dynamic range image, and relates to the technical field of high dynamic range image processing. The device comprises a first acquisition unit, a second acquisition unit, a first construction unit and a mapping unit. The first acquisition unit is used for acquiring HDR video data, and the HDR video data comprises a plurality of continuous frame HDR images. The second acquisition unit is used for acquiring feature values of each color feature corresponding to the HDR image based on a plurality of color features. The first construction unit is used for constructing a tone mapping index composed of the plurality of color features by assigning different weights to each feature value. The mapping unit is used for mapping the HDR image on a display device based on the tone mapping index to output a display picture. The application is applied to optimizing the picture quality obtained by tone mapping of the image.

Description

Technical Field

[0001] The present application relates to the technical field of high dynamic range image processing, and in particular to a tone mapping device and method for high dynamic range images. Background Art

[0002] High Dynamic Range (HDR) is a technology used to achieve a wider dynamic range (i.e., greater differences between light and dark) than conventional digital imaging techniques. While traditional image acquisition and processing processes can degrade the quality of the original footage, HDR technology reduces this loss and expands the display range of display devices. This results in HDR images with richer colors, brighter highlights, darker shadows, and improved contrast compared to traditional Standard Dynamic Range (SDR) images, providing a superior viewing experience.

[0003] Currently, when displaying HDR images based on tone mapping, the HDR image is mapped to display devices of different brightness levels using a luminance reference point to present the display image. However, if the luminance reference point used is not appropriate, the displayed image will suffer from image quality loss. Summary of the Invention

[0004] The present application provides a tone mapping device and method for high dynamic range images, the main purpose of which is to optimize the image quality obtained by tone mapping of HDR images and improve the image quality performance.

[0005] In order to achieve the above objectives, this application mainly provides the following technical solutions:

[0006] In a first aspect, the present application provides a tone mapping device for a high dynamic range image, the device comprising:

[0007] a first acquiring unit, a second acquiring unit, a first constructing unit, and a mapping unit; the first acquiring unit is connected to the second acquiring unit, the second acquiring unit is connected to the first constructing unit, and the first constructing unit is connected to the mapping unit;

[0008] The first acquisition unit is configured to acquire HDR video data, where the HDR video data includes a plurality of consecutive frame HDR images;

[0009] The second acquisition unit is configured to acquire a feature value of each color feature corresponding to the HDR image based on the multiple color features;

[0010] The first construction unit is configured to construct a tone mapping index composed of a plurality of the color features by assigning a different weight to each of the feature values;

[0011] The mapping unit is configured to map the HDR image to a display device based on the tone mapping index to output a display picture.

[0012] In some modified implementations of the first aspect of the present application, the device further includes:

[0013] a dividing unit, configured to, after acquiring the HDR video data, divide the HDR image contained in the HDR video data into a plurality of groups of image frames, each group of image frames containing a plurality of consecutive HDR image frames;

[0014] a third acquiring unit, configured to acquire characteristic elements corresponding to each group of image frames by parsing each group of image frames, wherein the characteristic elements are used to characterize a color type of the image frame group;

[0015] The fourth acquiring unit is configured to acquire the color feature combination strategy corresponding to the feature element by searching for a pre-established mapping relationship between the preset feature element and the preset color feature combination strategy.

[0016] In some modified implementations of the first aspect of the present application, the second acquiring unit includes:

[0017] A first acquisition module is configured to acquire at least one corresponding color feature from the color feature combination strategy corresponding to the feature element as the color feature associated with the corresponding image frame group;

[0018] A processing module is used to process each HDR image in each group of image frames based on at least one of the color features to obtain a feature value of the color feature corresponding to each HDR image.

[0019] In some modified implementations of the first aspect of the present application, the processing module includes:

[0020] a parsing submodule, configured to parse attribute information corresponding to each color feature from each HDR image in the image frame group;

[0021] The processing submodule is used to perform normalization processing on the attribute information corresponding to each color feature to obtain the characteristic value of the color feature corresponding to the HDR image.

[0022] In some modified embodiments of the first aspect of the present application, the first building block includes:

[0023] A second acquisition module is configured to acquire a weight curve corresponding to a color feature combination strategy by searching a pre-established mapping relationship between the preset color feature combination strategy and the preset weight curve;

[0024] The second acquisition module is further configured to acquire at least one corresponding color feature from the color feature combination strategy as the color feature associated with the corresponding image frame group;

[0025] a determination module, configured to determine a weight corresponding to each color feature by searching the weight curve based on the color features associated with the image frame group;

[0026] A construction module is used to construct a tone mapping index composed of multiple color features by using the product of the feature value corresponding to each color feature and the weight.

[0027] In some modified implementations of the first aspect of the present application, the device further includes:

[0028] a second construction unit configured to, if the preset color feature combination strategy includes two preset color features, construct a weight curve corresponding to the two preset color features based on a correlation between a product of characteristic values ​​corresponding to the two preset color features and a target weight coefficient, wherein the target weight coefficient is used to represent a weight assigned to one of the two preset color features;

[0029] The determining unit is configured to use the weight curve as a preset weight curve corresponding to the preset color feature combination strategy.

[0030] A second aspect of the present application provides a tone mapping method for a high dynamic range image, the method comprising:

[0031] Acquire HDR video data, where the HDR video data includes a plurality of consecutive frame HDR images;

[0032] Based on the plurality of color features, obtaining a feature value of each color feature corresponding to the HDR image;

[0033] Constructing a tone mapping index composed of a plurality of the color features by assigning a different weight to each of the feature values;

[0034] Based on the tone mapping index, the HDR image is mapped to a display device to output a display picture.

[0035] In some modified implementations of the second aspect of the present application, after obtaining the HDR video data, the method further includes:

[0036] Dividing the HDR images contained in the HDR video data into a plurality of groups of image frames, each group of image frames containing a plurality of consecutive HDR image frames;

[0037] By parsing each group of image frames, characteristic elements corresponding to each group of image frames are obtained, wherein the characteristic elements are used to characterize the color type of the image frame group;

[0038] By searching for a pre-established mapping relationship between a preset feature element and a preset color feature combination strategy, the color feature combination strategy corresponding to the feature element is obtained.

[0039] In some modified implementations of the second aspect of the present application, obtaining a feature value of each color feature corresponding to the HDR image based on multiple color features includes:

[0040] Acquire at least one corresponding color feature from the color feature combination strategy corresponding to the feature element as the color feature associated with the corresponding image frame group;

[0041] Based on at least one of the color features, each HDR image in each group of image frames is processed to obtain a feature value of the color feature corresponding to each HDR image.

[0042] In some modified implementations of the second aspect of the present application, processing each HDR image in the image frame group based on at least one of the color features to obtain a feature value of the color feature corresponding to each HDR image includes:

[0043] In the image frame group, parsing attribute information corresponding to each color feature from each HDR image;

[0044] Normalization is performed on the attribute information corresponding to each color feature to obtain a feature value of the color feature corresponding to the HDR image.

[0045] In some modified implementations of the second aspect of the present application, constructing a tone mapping index composed of multiple color features by assigning a different weight to each feature value includes:

[0046] Obtaining a weight curve corresponding to the color feature combination strategy by searching for a mapping relationship between a pre-established preset color feature combination strategy and a preset weight curve;

[0047] Acquire at least one corresponding color feature from the color feature combination strategy as the color feature associated with the corresponding image frame group;

[0048] Based on the color features associated with the image frame group, determining the weight corresponding to each color feature by searching the weight curve;

[0049] A tone mapping index composed of a plurality of the color features is constructed by multiplying the feature value corresponding to each of the color features by a weight.

[0050] In some modified implementations of the second aspect of the present application, the method further includes:

[0051] If the preset color feature combination strategy includes two preset color features, then based on the correlation between the product of the feature values ​​corresponding to the two preset color features and the target weight coefficient, a weight curve corresponding to the two preset color features is constructed, wherein the target weight coefficient is used to represent the weight assigned to one of the two preset color features;

[0052] The weight curve is used as the preset weight curve corresponding to the preset color feature combination strategy.

[0053] A third aspect of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for tone mapping a high dynamic range image as described above is implemented.

[0054] In a fourth aspect, the present application provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the tone mapping method for high dynamic range images as described above is implemented.

[0055] By means of the above technical solution, the technical solution provided by this application has at least the following advantages:

[0056] The present application provides a tone mapping device and method for high dynamic range images. The device provided by the present application includes: a first acquisition unit, a second acquisition unit, a first construction unit, and a mapping unit. The present application obtains multiple continuous frame HDR images based on the HDR video data obtained by the first acquisition unit, so that for each HDR image, the second acquisition unit is used to obtain the characteristic values ​​corresponding to the multiple color features, and the first construction unit is used to assign different weights to each characteristic value to construct a tone mapping index composed of multiple color features. The mapping unit is used to map this HDR image to a display device based on the tone mapping index to output a display screen. Therefore, based on the device provided by the present application, tone mapping processing of each HDR image based on the tone mapping index composed of multiple color features is realized. Compared with the existing technology, the problem of image quality loss caused by relying solely on the brightness reference base point to achieve tone mapping is solved. The present application optimizes the image quality obtained by tone mapping the HDR image and improves the image quality performance.

[0057] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present application. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0059] Figure 1 A block diagram of a tone mapping device for high dynamic range images is provided for an embodiment of the present application;

[0060] Figure 2 A block diagram of another high dynamic range image tone mapping device is provided for an embodiment of the present application;

[0061] Figure 3 A schematic diagram of a preset weight curve corresponding to a combination strategy consisting of brightness features and saturation features exemplified in an embodiment of the present application;

[0062] Figure 4 A flow chart of a tone mapping method for high dynamic range images provided in an embodiment of the present application;

[0063] Figure 5 A flowchart of another tone mapping method for high dynamic range images provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] The following describes exemplary embodiments of the present application in more detail with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0065] The embodiment of the present application provides a tone mapping device for high dynamic range images, such as Figure 1 As shown, the device includes: a first acquisition unit 11, a second acquisition unit 12, a first construction unit 13 and a mapping unit 14; the first acquisition unit 11 is connected to the second acquisition unit 12, the second acquisition unit 12 is connected to the construction unit 13, and the first construction unit 13 is connected to the mapping unit 14.

[0066] The first acquisition unit 11 is used to acquire HDR video data, which includes multiple consecutive frame HDR images. In the embodiment of the present application, the HDR video data is derived from a captured high dynamic range (HDR) image, and multiple consecutive frame HDR images can be obtained by analyzing the image.

[0067] The second acquiring unit 12 is configured to acquire a feature value of each color feature corresponding to the HDR image based on the multiple color features.

[0068] The color feature refers to widely used general features such as brightness, saturation, and hue, or may be some custom features, such as statistical value features. In the embodiment of the present application, for any color feature, each pixel in the image is used as the minimum processing level, and the feature value of the color feature of each pixel is obtained and further correlation operations are performed to obtain the feature value corresponding to the entire image.

[0069] Example 1, taking brightness features as an example, for the embodiments of the present application, the first step is to obtain the characteristic value of the brightness feature of each pixel in the image, and then perform correlation operations based on this to obtain the corresponding characteristic value of the brightness feature of the entire image.

[0070] Example 2, explaining the statistical value feature. For example, an HDR image is divided into multiple pixel blocks (10*10), and the processing for any pixel block is: obtain the component value of each pixel in the red (R) channel (referred to as R value), and calculate the average value (Rav) according to the R value corresponding to each pixel. Then there is a difference between the R value of each pixel and Rav. These discrete differences are accumulated to obtain a statistical value as the statistical value feature of the discrete degree of the R channel component value, and accordingly, this statistical value is used as the characteristic value corresponding to the pixel block. For this statistical value feature, since the image is divided into multiple pixel blocks, the characteristic value corresponding to the color feature of the "discrete degree of the R channel component value" of the entire image can be constructed based on the characteristic value combination corresponding to each pixel block.

[0071] It's important to note that for any pixel block, a larger statistical value indicates a greater dispersion between the R values ​​of those pixels, indicating that there are pixels with significant differences in brightness on the R channel within that block. Conversely, a smaller statistical value indicates a smaller dispersion, indicating that the brightness differences between the pixels within that block on the R channel are relatively small. Therefore, when tone mapping each pixel block in an image, the mapping parameters should be adaptively adjusted to account for the different eigenvalues ​​of each pixel block to avoid image quality loss.

[0072] The first constructing unit 13 is configured to construct a tone mapping index consisting of multiple color features by assigning different weights to each feature value.

[0073] Among them, the format of HDR image is HDRI file, which is a file with the extension of hdr or tif format. HDR image records the brightness value of the actual scene far beyond 256 levels. The excess part cannot be displayed on the display device screen. Since most display devices still use a maximum display brightness of 100nit, before displaying the actual scene exceeding 256 brightness levels using ordinary display devices, some conversion of brightness and chromaticity and other aspects is required to map the HDR image into a low dynamic range image. This process is called tone mapping.

[0074] In the embodiment of the present application, during the tone mapping process, a tone mapping index is constructed based on multiple color features, rather than relying solely on a luminance reference point to perform a chromaticity mapping operation.

[0075] For example, before constructing the chromaticity mapping index, multiple color features can be pre-selected, and for each HDR image, the eigenvalues ​​corresponding to each of these color features are obtained. Based on the size of these eigenvalues, different weights can be assigned, so as to construct a chromaticity mapping index using the eigenvalues ​​and weights corresponding to each of these color features.

[0076] For example, taking brightness and saturation features as examples, the constructed chromaticity mapping indicators are as follows:

[0077] Chroma mapping index = weight 1 * brightness eigenvalue + weight 2 * saturation eigenvalue; formula (1);

[0078] The mapping unit 14 is configured to map the HDR image to a display device based on a tone mapping index to output a display image.

[0079] In the embodiment of the present application, the following tone mapping formula may be used, namely formula (2):

[0080] The tone mapping formula is as follows: Output tone = tone mapping function (input tone); Formula (2);

[0081] The tone mapping function can be linear or nonlinear, with linear mapping being used as an example. Output tone = tone gain (chromaticity mapping index) * input tone. Using a preset tone gain curve, the "tone mapping index" is used to find the corresponding "tone gain" and multiply it by the "input tone" to complete the tone mapping, yielding the output tone. Based on the output tone, the HDR image is then mapped to the display device for display.

[0082] An embodiment of the present application provides a tone mapping device for a high dynamic range image. The device provided by the present application includes: a first acquisition unit 11, a second acquisition unit 12, a first construction unit 13, and a mapping unit 14. Based on the HDR video data acquired by the first acquisition unit 11, the present embodiment obtains multiple consecutive frames of HDR images. For each HDR image, the second acquisition unit 12 is used to acquire feature values ​​corresponding to multiple color features. The first construction unit 13 is used to assign different weights to each feature value to construct a tone mapping index composed of multiple color features. The mapping unit 14 then maps the HDR image to a display device based on the tone mapping index to output a display screen. Therefore, based on the device provided by the present application, tone mapping processing is implemented for each HDR image based on the tone mapping index composed of multiple color features. Compared to the prior art, this solves the problem of image quality loss caused by relying solely on a brightness reference point to implement tone mapping. The embodiment of the present application optimizes the image quality of the HDR image obtained through tone mapping, thereby improving the image quality performance.

[0083] In some modified embodiments, the present application also provides another tone mapping device for high dynamic range images, such as Figure 2 As shown, the device provided in the above embodiment is further refined to supplement more functional applications.

[0084] like Figure 2 As shown, the tone mapping device for a high dynamic range image provided in an embodiment of the present application includes, in addition to the first acquisition unit 11, the second acquisition unit 12, the first construction unit 13 and the mapping unit 14, the device also includes:

[0085] The dividing unit 15 is used to divide the HDR image contained in the HDR video data into multiple groups of image frames after obtaining the HDR video data, each group of image frames contains multiple consecutive frame HDR images, that is, a group of image frames can also be called "an image frame group"; the third obtaining unit 16 is used to obtain the characteristic elements corresponding to each group of image frames by parsing each group of image frames, and the characteristic elements are used to characterize the color type of the image frame group; the fourth obtaining unit 17 is used to obtain the color feature combination strategy corresponding to the characteristic element by searching for the mapping relationship between the pre-established preset characteristic elements and the preset color feature combination strategy, wherein each preset color feature combination strategy pre-includes different preset color features.

[0086] Among them, for the embodiment of the present application, the characteristic element can be a color type, such as but not limited to bright color, dull color, single color or rich color, etc.

[0087] Among them, the embodiments of the present application can pre-set multiple color feature combination strategies for application to different example scenarios. For example, for landscape scenes with a heavy black and white tone, because HDR images carry fewer other colors, a preset color feature combination strategy can be constructed using brightness features and saturation features; while for bright and colorful landscape scenes, a preset color feature combination strategy can be constructed using brightness features, saturation features, and hue features. Of course, for the constructed preset color feature combination strategy, custom features can also be added according to the actual scene requirements.

[0088] In an embodiment of the present application, different preset feature elements can be used to refer to these instance scenarios. For example, the preset feature elements can be color types, such as but not limited to bright colors, dim colors, single colors or rich colors, etc., so that a mapping relationship between the preset feature elements and the preset color feature combination strategies can be established in advance, so that the best color feature combination strategy can be selected based on the mapping relationship for different instance scenarios.

[0089] In an embodiment of the present application, the multiple continuous frame HDR images contained in the acquired HDR video data can be divided into multiple groups of image frames. Specifically, the division method can be but is not limited to an equal division method, an unequal division method, or a combination of the two.

[0090] Furthermore, by parsing each group of image frames, characteristic elements corresponding to each group of image frames are obtained. A specific implementation method may include, but is not limited to, first obtaining characteristic elements of each image within the image frame group and then, based on similarity comparison, determining characteristic elements that are common to the image frame group as the characteristic elements corresponding to the image frame group. For example, after processing the image frame group, the color type of the image frame group is determined to be "landscape scene with a predominantly black and white hue."

[0091] Preferably, standardized feature elements can be pre-defined. Using these pre-defined standardized feature elements as a benchmark, the image frame group is determined to match the image frame group, thereby obtaining the feature elements corresponding to the image frame group. This approach aims to avoid the diversity of feature elements, which makes it difficult to accurately determine which feature element the image frame group belongs to, making it difficult to subsequently find a color feature combination strategy that is suitable for the image frame group.

[0092] It should be noted that the embodiment of the present application hopes to divide the image represented by the original HDR video data into multiple small and different instance scenes through a division operation, so as to flexibly match different preset color feature combination strategies to process the image frame group corresponding to the small instance scene. Accordingly, when performing the division operation on the HDR image contained in the HDR video data into multiple image frame groups, it is necessary to consider that the number of image frames in each group is sufficient to enable the image frame group to clearly express the characteristic elements, and it is also necessary to consider that the characteristic elements expressed by the two adjacent image frame groups should be different, so as to obtain two different small instance scenes.

[0093] like Figure 2 As shown, the second acquisition unit 12 is used to obtain the feature value corresponding to each color feature from the HDR image based on multiple color features. The second acquisition unit 12 is further divided into:

[0094] The first acquisition module 121 is configured to acquire at least one corresponding color feature from the color feature combination strategy corresponding to the feature element as the color feature associated with the corresponding image frame group.

[0095] The processing module 122 is configured to process each HDR image in the image frame group based on at least one color feature to obtain a feature value of the color feature corresponding to each HDR image.

[0096] In the embodiment of the present application, instead of directly selecting color features for each HDR image to be processed, a matching color feature combination strategy is found for the feature elements corresponding to each image frame group, and the color features corresponding to the color feature combination strategy are used to process each HDR image in the entire image frame group. It should be noted that if the feature elements corresponding to different image frame groups are different, the matching color feature combination strategies are found accordingly, thereby achieving the selection of the best color features for processing for each image frame group, rather than just using general color features for processing, thereby meeting the flexible processing requirements for HDR images.

[0097] like Figure 2 As shown, the processing module 122 can be further divided into:

[0098] The parsing submodule 1221 is used to parse the attribute information corresponding to each color feature from each HDR image in the image frame group;

[0099] The processing submodule 1222 is used to perform normalization processing on the attribute information corresponding to each color feature to obtain a feature value of the color feature corresponding to the HDR image.

[0100] In an embodiment of the present application, taking the color feature combination strategy consisting of brightness features and saturation features as an example, when processing each HDR image in an image frame group, attribute information corresponding to the brightness feature and attribute information corresponding to the saturation feature are parsed from each HDR image, wherein the attribute information corresponding to the brightness feature of the image refers to data information composed of the brightness value of each pixel, and the attribute information corresponding to the saturation feature of the image refers to data information composed of the saturation value of each pixel. The purpose of the normalization processing in the embodiment of the present application is to convert the attribute information of the color feature into a value between 0 and 1.

[0101] For example, taking the brightness feature as an example, for the attribute information corresponding to the brightness feature of the image, the brightness value of each pixel is first normalized, and then the brightness value of the entire image is normalized based on this. Specifically, the brightness value of a pixel is normalized using the following formula (3):

[0102] Brightness characteristic value = pixel brightness value / maximum value of brightness quantization value; Formula (3);

[0103] The explanation is: Taking 8-bit as an example, the input brightness is 128, and the maximum quantized value is 255. The normalization process is 128 / 255=0.5019..., which is rounded to the second decimal place to 0.50, which is used as the eigenvalue of the brightness feature of the pixel. Based on this eigenvalue, the brightness of a pixel is measured as high, medium or low.

[0104] Furthermore, in an embodiment of the present application, for brightness features, an averaging operation is performed based on the feature values ​​of each pixel point to obtain the feature value corresponding to the entire image, which serves as a normalized processing result of the attribute information of the brightness feature corresponding to the image.

[0105] like Figure 2 As shown, the construction unit 13 is used to construct a tone mapping index composed of multiple color features by assigning different weights to each feature value. The construction unit 13 can be further divided into:

[0106] The second acquisition module 131 is used to obtain the weight curve corresponding to the color feature combination strategy by searching for the mapping relationship between the pre-established preset color feature combination strategy and the preset weight curve; the acquisition module 131 is also used to obtain at least one corresponding color feature from the color feature combination strategy as the color feature associated with the corresponding image frame group; the determination module 132 is used to determine the weight corresponding to each color feature by searching the weight curve based on the color feature associated with the image frame group; the construction module 133 is used to construct a tone mapping index composed of multiple color features using the product of the characteristic value corresponding to each color feature and the weight.

[0107] In an embodiment of the present application, a mapping relationship between multiple preset color feature combination strategies and preset weight curves can be pre-set, so that for the color feature combination strategy that matches the characteristic elements of an image frame group, the corresponding weight curve can be obtained by further searching the mapping relationship, so as to further realize the assignment of different weights to each color feature corresponding to the color feature combination strategy based on the curve.

[0108] For example, still taking the color feature combination strategy composed of brightness feature and saturation feature as an example, by searching the mapping relationship between the pre-established preset color feature combination strategy and the preset weight curve, the weight curve matching the combination strategy composed of these two color features is obtained, such as Figure 3 As shown: the horizontal axis is the product of the brightness eigenvalue and the saturation eigenvalue. Since the eigenvalue is a normalized value, the maximum product of the two is 1; the vertical axis is the weight, and the maximum value is 1. For example, the weight corresponding to the saturation eigenvalue can be assigned. Therefore, the following tone mapping indicator formula is adopted:

[0109] Tone mapping index = (1-k)*brightness eigenvalue + k*saturation eigenvalue; Formula (4);

[0110] Among them, the value range of k is [0,1], such as Figure 3 As shown in the curve, assuming that the higher the brightness eigenvalue and the higher the saturation eigenvalue, the horizontal axis approaches 1, and k takes the value of 0.5, then the tone mapping index is composed of a brightness eigenvalue of 0.5 and a saturation eigenvalue of 0.5. Such a tone mapping index is used to optimize the tone mapping operation parameters to avoid image quality loss of the display device.

[0111] In the embodiment of the present application, for the color feature combination strategy consisting of brightness feature and saturation feature, based on the product of brightness feature value and saturation feature value, find the following: Figure 3 The weight curve shown in FIG1 can obtain the value of k, thereby substituting k into the above formula (4) to obtain the tone mapping index applicable to the color feature combination strategy, which is used to perform tone mapping processing on the corresponding image frame group (i.e., the image frame group applicable to the color feature combination strategy) based on the tone mapping index.

[0112] like Figure 2 As shown, the device provided in the embodiment of the present application also includes:

[0113] The second construction unit 18 is configured to construct a weight curve corresponding to the two preset color features based on a correlation between the product of the characteristic values ​​corresponding to the two preset color features and a target weight coefficient, if the preset color feature combination strategy includes two preset color features. The target weight coefficient is used to represent the weight assigned to one of the two preset color features. The determination unit 19 is configured to use the weight curve as the preset weight curve corresponding to the preset color feature combination strategy consisting of the two preset color features.

[0114] For an example scenario in which the preset color feature combination strategy includes two preset color features, the embodiment of the present application uses the second construction unit 18 and the determination unit 19 to provide an implementation process for constructing the corresponding preset weight curve.

[0115] For example, taking brightness and saturation features as an example, for a tone mapping index composed of these two color features, if the brightness changes during tone mapping processing, it will also affect the saturation presentation effect. For example, from the perspective of the human eye, the brighter the place, the less clear the color will be or even whitish, that is, the higher the brightness, the lower the relative saturation. Therefore, when constructing a tone mapping index composed of these two color features, the effect of the two color features working together should be considered when assigning weights to the color features.

[0116] Therefore, in the embodiment of the present application, the weight curves corresponding to the two preset color features can be constructed based on the correlation between the product of the corresponding feature values ​​of the two preset color features and the target weight coefficient, such as Figure 3 Preset weight curves shown.

[0117] It should be noted that if Figure 3 As shown, the vertical coordinate k refers to the "target weight coefficient", which is used to represent the weight assigned to one of the two preset color features, for example, as the weight assigned to the saturation feature, so as to construct the tone mapping index such as the above formula (4) based on such a "target weight coefficient".

[0118] Furthermore, as a response to the above Figure 1 The present invention provides a method for tone mapping of high dynamic range images by using the device shown in the figure. This method embodiment corresponds to the aforementioned device embodiment. For ease of reading, this method embodiment will not repeat the details of the aforementioned device embodiment one by one, but it should be clear that the method in this embodiment can correspond to all the contents of the aforementioned device embodiment. Figure 4 As shown, the embodiment of this application provides the following specific steps:

[0119] 201. Acquire HDR video data, where the HDR video data includes multiple consecutive HDR image frames.

[0120] 202. Based on the multiple color features, obtain a feature value of each color feature corresponding to the HDR image.

[0121] 203. A tone mapping index consisting of multiple color features is constructed by assigning different weights to each feature value.

[0122] 204. Based on the tone mapping index, map the HDR image to a display device to output a display image.

[0123] Furthermore, as a response to the above Figure 2 The present invention provides another method for tone mapping of high dynamic range images by using the device shown in the figure. This method embodiment corresponds to the above-mentioned device embodiment. For ease of reading, this method embodiment will not repeat the details of the above-mentioned device embodiment one by one, but it should be clear that the method in this embodiment can correspond to all the contents of the above-mentioned device embodiment. Figure 5 As shown, the embodiment of this application provides the following specific steps:

[0124] 301. Divide the HDR images included in the HDR video data into multiple groups of image frames, each group of image frames including multiple consecutive HDR image frames.

[0125] 302. Obtain characteristic elements corresponding to each group of image frames by parsing each group of image frames, where the characteristic elements are used to represent the color type of the image frame group.

[0126] 303. Obtain a color feature combination strategy corresponding to the feature element by searching for a pre-established mapping relationship between the preset feature element and the preset color feature combination strategy, wherein each preset color feature combination strategy pre-includes different preset color features.

[0127] 304. Acquire HDR video data, where the HDR video data includes multiple consecutive HDR image frames.

[0128] 305 . Obtain at least one corresponding color feature from the color feature combination strategy corresponding to the feature element as the color feature associated with the corresponding image frame group.

[0129] 306. Process each HDR image in the image frame group based on at least one color feature to obtain a feature value of the color feature corresponding to each HDR image.

[0130] This step can be explained in detail as follows: in the image frame group, the attribute information corresponding to each color feature is parsed from each HDR image; the attribute information corresponding to each color feature is normalized to obtain the characteristic value of the color feature corresponding to the HDR image.

[0131] 307 : Acquire a weight curve corresponding to the color feature combination strategy by searching for a mapping relationship between a pre-established preset color feature combination strategy and a preset weight curve.

[0132] The embodiment of the present application cites an application scenario in which the preset color feature combination strategy includes two color features. This step can be explained in detail as follows:

[0133] If the preset color feature combination strategy includes two preset color features, a weight curve corresponding to the two preset color features is constructed based on the correlation between the product of the feature values ​​corresponding to the two preset color features and the target weight coefficient, where the target weight coefficient is used to represent the weight assigned to one of the two preset color features. The weight curve is then used as the preset weight curve corresponding to the preset color feature combination strategy composed of the two preset color features.

[0134] 308. Obtain at least one corresponding color feature from the color feature combination strategy as the color feature associated with the corresponding image frame group.

[0135] 309. Based on the color features associated with the image frame group, determine the weight corresponding to each color feature by searching a weight curve.

[0136] 310. A tone mapping index composed of multiple color features is constructed by multiplying the eigenvalue corresponding to each color feature by its weight.

[0137] 311. Based on the tone mapping index, map the HDR image to a display device to output a display image.

[0138] To sum up, the embodiments of the present application provide a tone mapping device and method for high dynamic range images. Using the device provided by the embodiments of the present application, for the multiple continuous frame HDR images contained in the acquired HDR video data, the embodiments of the present application divide them into multiple image frame groups, so as to find a matching color feature combination strategy for each image frame group and obtain corresponding multiple color features, so as to process each HDR image in the image frame group based on these color features. The embodiments of the present application select the best matching color feature combination strategy for different image frame groups, and further find the best matching weight curve based on the matching color feature combination strategy, so as to achieve flexible processing of image frame groups with different feature elements.

[0139] Furthermore, for the HDR images within the image frame group, characteristic values ​​corresponding to the color features are obtained from each image, and different weights are assigned to each characteristic value based on a weight curve, to construct a tone mapping index composed of multiple color features. The tone mapping index is used to map the HDR image to a display device to output a display screen, thereby completing the mapping processing of each HDR image within an image frame group, and thereby completing the mapping processing of each image frame group. Accordingly, the embodiment of the present application can flexibly select multiple color features to form the best color feature combination strategy for the tone mapping index, thereby achieving tone mapping processing for each HDR image, thereby avoiding image quality loss as much as possible, thereby optimizing the image quality of the HDR image obtained by tone mapping, and improving the image quality performance.

[0140] The tone mapping device for high dynamic range images provided in an embodiment of the present application includes a processor and a memory. The above-mentioned first acquisition unit, second acquisition unit, construction unit and mapping unit are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0141] The processor contains a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set. By adjusting the kernel parameters, the image quality of the HDR image obtained by tone mapping is optimized, improving the image quality performance.

[0142] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for tone mapping a high dynamic range image as described above is implemented.

[0143] An embodiment of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the tone mapping method for high dynamic range images as described above when executing the computer program.

[0144] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0145] In a typical configuration, the device includes one or more processors (CPUs), memory, and a bus. The device may also include input / output interfaces, network interfaces, and the like.

[0146] Memory may include non-permanent memory in a computer-readable medium, random access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory includes at least one memory chip. Memory is an example of a computer-readable medium.

[0147] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0148] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0149] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0150] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A tone mapping device for high dynamic range images, characterized in that: The device includes: a first acquisition unit, a second acquisition unit, a first construction unit and a mapping unit; the first acquisition unit is connected to the second acquisition unit, the second acquisition unit is connected to the first construction unit, and the first construction unit is connected to the mapping unit; The first acquisition unit is configured to acquire HDR video data, where the HDR video data includes a plurality of consecutive frame HDR images; a dividing unit, configured to, after acquiring the HDR video data, divide the HDR image contained in the HDR video data into a plurality of groups of image frames, each group of image frames containing a plurality of consecutive HDR image frames; a third acquiring unit, configured to acquire characteristic elements corresponding to each group of image frames by analyzing each group of image frames, wherein the characteristic elements are used to characterize a color type of the image frame group, wherein the color type is bright color, dull color, single color, or rich color; a fourth acquiring unit, configured to acquire a color feature combination strategy corresponding to a feature element by searching a pre-established mapping relationship between a preset feature element and a preset color feature combination strategy, wherein the color feature is at least a brightness feature, a saturation feature, or a hue feature; The second acquisition unit is configured to acquire a feature value of each color feature corresponding to the HDR image based on the multiple color features; The second acquisition unit includes: a first acquisition module, configured to acquire at least one corresponding color feature from the color feature combination strategy corresponding to the feature element as the color feature associated with the corresponding image frame group; a processing module, configured to process each HDR image in each group of image frames based on the at least one color feature to obtain a feature value of the color feature corresponding to each HDR image; The first construction unit is configured to construct a tone mapping index composed of a plurality of the color features by assigning a different weight to each of the feature values; The mapping unit is configured to map the HDR image to a display device based on the tone mapping index to output a display picture.

2. The device according to claim 1, characterized in that The processing module includes: a parsing submodule, configured to parse attribute information corresponding to each color feature from each HDR image in the image frame group; The processing submodule is used to perform normalization processing on the attribute information corresponding to each color feature to obtain the characteristic value of the color feature corresponding to the HDR image.

3. The device according to claim 1, characterized in that The first building block comprises: A second acquisition module is configured to acquire a weight curve corresponding to a color feature combination strategy by searching a pre-established mapping relationship between the preset color feature combination strategy and the preset weight curve; The second acquisition module is further configured to acquire at least one corresponding color feature from the color feature combination strategy as the color feature associated with the corresponding image frame group; a determination module, configured to determine a weight corresponding to each color feature by searching the weight curve based on the color features associated with the image frame group; A construction module is used to construct a tone mapping index composed of multiple color features by using the product of the feature value corresponding to each color feature and the weight.

4. The device according to claim 3, characterized in that The device further comprises: a second construction unit configured to, if the preset color feature combination strategy includes two preset color features, construct a weight curve corresponding to the two preset color features based on a correlation between a product of characteristic values ​​corresponding to the two preset color features and a target weight coefficient, wherein the target weight coefficient is used to represent a weight assigned to one of the two preset color features; The determining unit is configured to use the weight curve as a preset weight curve corresponding to the preset color feature combination strategy.

5. A tone mapping method for high dynamic range images, characterized in that: The method comprises: Acquire HDR video data, where the HDR video data includes a plurality of consecutive frame HDR images; Dividing the HDR images contained in the HDR video data into a plurality of groups of image frames, each group of image frames containing a plurality of consecutive HDR image frames; By parsing each group of image frames, characteristic elements corresponding to each group of image frames are obtained, wherein the characteristic elements are used to characterize the color type of the image frame group, and the color type is bright color, dull color, single color or rich color; By searching for a pre-established mapping relationship between a preset feature element and a preset color feature combination strategy, a color feature combination strategy corresponding to the feature element is obtained, where the color feature is at least a brightness feature, a saturation feature, or a hue feature; Obtaining, based on the plurality of color features, characteristic values ​​of respective color features corresponding to the HDR image, including: obtaining at least one corresponding color feature from the color feature combination strategy corresponding to the characteristic element as a color feature associated with the corresponding image frame group; processing each HDR image in each group of image frames based on the at least one color feature to obtain characteristic values ​​of the color features corresponding to each HDR image; Constructing a tone mapping index composed of a plurality of the color features by assigning a different weight to each of the feature values; Based on the tone mapping index, the HDR image is mapped to a display device to output a display picture.

6. The method according to claim 5, characterized in that The processing of each HDR image in the image frame group based on at least one of the color features to obtain a feature value of the color feature corresponding to each HDR image includes: In the image frame group, parsing attribute information corresponding to each color feature from each HDR image; Normalization is performed on the attribute information corresponding to each color feature to obtain a feature value of the color feature corresponding to the HDR image.

7. The method according to claim 5, characterized in that The step of constructing a tone mapping index composed of a plurality of the color features by assigning a different weight to each of the feature values ​​comprises: Obtaining a weight curve corresponding to the color feature combination strategy by searching for a mapping relationship between a pre-established preset color feature combination strategy and a preset weight curve; Acquire at least one corresponding color feature from the color feature combination strategy as the color feature associated with the corresponding image frame group; Based on the color features associated with the image frame group, determining the weight corresponding to each color feature by searching the weight curve; A tone mapping index composed of a plurality of the color features is constructed by multiplying the feature value corresponding to each of the color features by a weight.

8. The method according to claim 7, characterized in that The method further comprises: If the preset color feature combination strategy includes two preset color features, then based on the correlation between the product of the feature values ​​corresponding to the two preset color features and the target weight coefficient, a weight curve corresponding to the two preset color features is constructed, wherein the target weight coefficient is used to represent the weight assigned to one of the two preset color features; The weight curve is used as the preset weight curve corresponding to the preset color feature combination strategy.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the tone mapping method for a high dynamic range image according to any one of claims 5 to 8.

10. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the tone mapping method for a high dynamic range image according to any one of claims 5 to 8 when executing the computer program.

Images