A processing device and method for high dynamic range images based on tone mapping

Abstract

The application discloses a kind of processing device and method based on tone mapping of high dynamic range image, it is related to high dynamic range image processing technical field.The device of the present application includes: acquisition unit, for obtaining HDR video data, HDR video data contains multiple continuous frame HDR images;First processing unit, for carrying out channel processing to each HDR image, obtain the luminance channel image and color channel image corresponding to each HDR image respectively;Second processing unit is used to process the luminance channel image corresponding to HDR image using preset tone mapping algorithm, obtain first image;Third processing unit is used to process the color channel image corresponding to HDR image using preset color correction rule, obtain second image;Output unit is used to output the target display picture corresponding to each HDR image according to first image and second image.

Description

Technical Field

[0001] This application relates to the field of high dynamic range image processing technology, and in particular to a processing apparatus and method for high dynamic range images based on tone mapping. Background Technology

[0002] High Dynamic Range (HDR) is a technology used to achieve a greater dynamic range of exposure (i.e., a greater difference between light and dark areas) than ordinary digital imaging technology. Traditional image acquisition and processing processes lose the quality of the original material, while the HDR technology process can reduce this loss and expand the display range of display devices. This makes HDR images richer in color, brighter in highlight details, darker in shadow details, and have improved contrast compared to traditional Standard Dynamic Range (SDR) images, resulting in a better viewing experience.

[0003] Currently, in the process of displaying HDR images based on tone mapping, the HDR image is mainly mapped onto display devices with different brightness levels using a brightness reference point to present the displayed image. However, if only brightness is compressed, although the original colors are preserved, the image displayed on the device will be distorted due to color oversaturation, resulting in low image quality. Summary of the Invention

[0004] This application provides a processing apparatus and method for high dynamic range images based on tone mapping, the main purpose of which is to effectively correct color distortion after tone mapping of HDR images and improve the image quality of the output images of display devices.

[0005] To achieve the above objectives, this application mainly provides the following technical solutions:

[0006] The first aspect of this application provides a high dynamic range image processing apparatus based on tone mapping, the apparatus comprising:

[0007] The acquisition unit is used to acquire HDR video data, wherein the HDR video data contains multiple consecutive frame HDR images;

[0008] The first processing unit is used to perform channel-based processing on each of the HDR images to obtain the luminance channel image and color channel image corresponding to each of the HDR images.

[0009] The second processing unit is used to process the luminance channel image corresponding to the HDR image using a preset tone mapping algorithm to obtain the first image;

[0010] The third processing unit is used to process the color channel image corresponding to the HDR image using a preset color correction rule to obtain the second image;

[0011] The output unit is used to output the target display screen corresponding to each of the HDR images based on the first image and the second image.

[0012] In some modified embodiments of the first aspect of this application, the first processing unit includes:

[0013] The first acquisition module is used to acquire multiple pixels contained in the HDR image;

[0014] The parsing module is used to parse the luminance component value and the color component value from the pixel.

[0015] The first construction module is used to construct the luminance channel image corresponding to the HDR image based on the luminance component values ​​corresponding to each pixel.

[0016] The first construction module is further configured to construct a color channel image corresponding to the HDR image based on the color component values ​​corresponding to each pixel.

[0017] In some modified embodiments of the first aspect of this application, the apparatus further includes:

[0018] The partitioning unit is used to divide multiple consecutive HDR images into multiple image frame groups after performing channel-splitting processing on each of the HDR images to obtain the luminance channel image and color channel image corresponding to each of the HDR images. Each image frame group contains multiple adjacent HDR images.

[0019] The construction unit is used to construct a first channel image group and a second channel image group corresponding to each of the image frame groups, wherein the first channel image group contains a luminance channel image and the second channel image group contains a color channel image.

[0020] In some modified embodiments of the first aspect of this application, the building unit includes:

[0021] The second construction module is used to construct a first channel image group corresponding to the image frame group based on the brightness channel image corresponding to each of the HDR images in the image frame group.

[0022] The second construction module is further configured to construct a second channel image group corresponding to the image frame group based on the color channel images corresponding to each of the HDR images in the image frame group.

[0023] The association module is used to establish a mapping relationship between the first channel image group and the second channel image group based on the same image frame group.

[0024] In some modified embodiments of the first aspect of this application, the third processing unit includes:

[0025] The second acquisition module is used to acquire the target image that is in the center position from the second channel image group, wherein the position of each image in the second channel image group is the same as the position of the corresponding HDR image in the image frame group;

[0026] The determining module is used to use the target image as the feature image corresponding to the second channel image group;

[0027] The processing module is used to process the feature image using the preset color correction rules to obtain the corresponding color channel correction gain value;

[0028] The correction module is used to correct each color channel image in the second channel image group according to the color channel correction gain value, so as to obtain the second image corresponding to each color channel image.

[0029] In some modified embodiments of the first aspect of this application, the processing module includes:

[0030] The first acquisition submodule is used to acquire multiple pixels contained in the feature image;

[0031] The calculation submodule is used to calculate the feature value of the preset color feature corresponding to each pixel point based on the preset color feature;

[0032] The first determining submodule is used to determine the hue gain value corresponding to the feature value by finding the preset gain curve corresponding to the preset color feature;

[0033] The submodule is used to compose the color channel correction gain value corresponding to the feature image based on the preset color features and the hue gain value corresponding to the feature value of each pixel.

[0034] In some modified embodiments of the first aspect of this application, the computing submodule includes:

[0035] The second acquisition submodule is used to acquire the RGB value corresponding to each pixel.

[0036] The normalization processing submodule is used to normalize the RGB values ​​of each pixel based on a preset color feature to obtain the feature value of the preset color feature corresponding to each pixel.

[0037] In some modified embodiments of the first aspect of this application, if there are multiple preset color features, the constituent sub-modules include:

[0038] The allocation submodule is used to assign weights to multiple preset color features;

[0039] The second determining submodule is used to determine the hue gain value corresponding to the preset color feature based on the feature value of the preset color feature corresponding to each pixel.

[0040] A submodule is constructed to construct the color channel correction gain value corresponding to the feature image based on the weights corresponding to the multiple preset color features and the hue gain value corresponding to the preset color features.

[0041] A second aspect of this application provides a method for processing high dynamic range images based on tone mapping, the method comprising:

[0042] Acquire HDR video data, which contains multiple consecutive HDR images;

[0043] Each HDR image is processed by channel segmentation to obtain the luminance channel image and color channel image corresponding to each HDR image;

[0044] The luminance channel image corresponding to the HDR image is processed using a preset tone mapping algorithm to obtain a first image;

[0045] The color channel image corresponding to the HDR image is processed using a preset color correction rule to obtain a second image;

[0046] Based on the first image and the second image, output the target display image corresponding to each of the HDR images.

[0047] In some modified embodiments of the second aspect of this application, the step of performing channel-specific processing on each HDR image to obtain a luminance channel image and a color channel image corresponding to each HDR image includes:

[0048] Obtain multiple pixels contained in the HDR image;

[0049] The luminance component value and color component value are extracted from the pixel;

[0050] Based on the luminance component values ​​corresponding to each pixel, a luminance channel image corresponding to the HDR image is constructed;

[0051] Based on the color component values ​​corresponding to each pixel, a color channel image corresponding to the HDR image is constructed.

[0052] In some modified embodiments of the second aspect of this application, after performing channel-by-channel processing on each of the HDR images to obtain the luminance channel image and color channel image corresponding to each of the HDR images, the method further includes:

[0053] Multiple consecutive HDR images are divided into multiple image frame groups, and each image frame group contains multiple adjacent HDR images;

[0054] Construct a first channel image group and a second channel image group corresponding to each image frame group, wherein the first channel image group contains a luminance channel image and the second channel image group contains a color channel image.

[0055] In some modified embodiments of the second aspect of this application, constructing the first channel image group and the second channel image group corresponding to each image frame group includes:

[0056] Based on the brightness channel image corresponding to each HDR image in the image frame group, a first channel image group corresponding to the image frame group is constructed.

[0057] Based on the color channel images corresponding to each of the HDR images in the image frame group, a second channel image group corresponding to the image frame group is constructed.

[0058] Based on the same image frame group, a mapping relationship is established between the first channel image group and the second channel image group.

[0059] In some modified embodiments of the second aspect of this application, the step of processing the color channel image corresponding to the HDR image using a preset color correction rule to obtain a second image includes:

[0060] Obtain the target image that is centered from the second channel image group, wherein the ranking of each image in the second channel image group is the same as the ranking of the corresponding HDR image in the image frame group;

[0061] The target image is used as the feature image corresponding to the second channel image group;

[0062] The feature image is processed using the preset color correction rules to obtain the corresponding color channel correction gain value;

[0063] The color channel images in the second channel image group are corrected according to the color channel correction gain value to obtain the second image corresponding to each color channel image.

[0064] In some modified embodiments of the second aspect of this application, the step of processing the feature image using the preset color correction rule to obtain the corresponding color channel correction gain value includes:

[0065] Obtain multiple pixels contained in the feature image;

[0066] Based on preset color features, obtain the feature values ​​of the preset color features corresponding to each pixel;

[0067] The hue gain value corresponding to the feature value is determined by finding the preset gain curve corresponding to the preset color feature;

[0068] Based on the preset color features, the color channel correction gain value corresponding to the feature image is formed according to the hue gain value corresponding to the feature value of each pixel.

[0069] In some modified embodiments of the second aspect of this application, obtaining the feature value of the preset color feature corresponding to each pixel based on the preset color feature includes:

[0070] Obtain the RGB value corresponding to each pixel;

[0071] Based on preset color features, the RGB values ​​of each pixel are normalized to obtain the feature values ​​of the preset color features corresponding to each pixel.

[0072] In some modified embodiments of the second aspect of this application, if there are multiple preset color features, then the step of assembling the color channel correction gain value corresponding to the feature image based on the preset color features and the hue gain value corresponding to the feature value of each pixel includes:

[0073] Weights are assigned to multiple preset color features;

[0074] Based on the feature value of the preset color feature corresponding to each pixel, determine the hue gain value corresponding to the preset color feature;

[0075] The color channel correction gain value corresponding to the feature image is composed of the weights corresponding to each of the multiple preset color features and the hue gain value corresponding to the preset color features.

[0076] A third aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the high dynamic range image processing method based on tone mapping as described above.

[0077] A fourth aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the high dynamic range image processing method based on tone mapping as described above.

[0078] By employing the above-described technical solution, the technical solution provided in this application has at least the following advantages:

[0079] This application provides a processing apparatus and method for high dynamic range images based on tone mapping. The apparatus includes an acquisition unit, a first processing unit, a second processing unit, a third processing unit, and an output unit. The acquisition unit acquires HDR video data to obtain multiple consecutive frame HDR images. For each HDR image, the first processing unit performs channel-specific processing to obtain a luminance channel image and a color channel image. The second processing unit then performs tone mapping processing based on the luminance channel image to obtain a first image, and the third processing unit performs color correction processing based on the color channel image to obtain a second image. Therefore, the processing scheme provided by this application for HDR images separates luminance tone mapping and color correction, and then outputs the target display image based on the results of both processing using a display device. Compared to existing technologies, this solves the technical problem of color distortion and low image quality caused by only compressing luminance during tone mapping of HDR images. The luminance and color channel-specific processing scheme provided by this application is equivalent to performing color correction while performing luminance tone mapping on the HDR image, thereby maximizing the preservation of color quality without tone mapping loss and improving the image quality of the image output by the display device.

[0080] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0081] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0082] Figure 1 This application provides a block diagram of a high dynamic range image processing apparatus based on tone mapping.

[0083] Figure 2 The saturation correction gain curve is an example of an embodiment of this application;

[0084] Figure 3 This application provides a block diagram of another high dynamic range image processing apparatus based on tone mapping;

[0085] Figure 4 A flowchart illustrating a high dynamic range image processing method based on tone mapping, provided in an embodiment of this application;

[0086] Figure 5 A flowchart illustrating another high dynamic range image processing method based on tone mapping provided in this application embodiment. Detailed Implementation

[0087] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0088] This application provides a high dynamic range image processing apparatus based on tone mapping, such as... Figure 1 As shown, the device includes: an acquisition unit 11, a first processing unit 12, a second processing unit 13, a third processing unit 14, and an output unit 15.

[0089] The acquisition unit 11 is used to acquire HDR video data, which includes multiple consecutive HDR images. In this embodiment, the HDR video data originates from captured High Dynamic Range (HDR) images, and multiple consecutive HDR images can be obtained by analyzing these images.

[0090] The first processing unit 12 is used to perform channel-specific processing on each HDR image to obtain the luminance channel image and color channel image corresponding to each HDR image.

[0091] In this embodiment of the application, for example, the YUV color space can be used to process each pixel in the HDR image by channel, where "Y" represents luminance (or Luma), which is the grayscale value; while "U" and "V" represent chrominance (or Chroma), which is the property of color excluding luminance and is used to specify the color of a pixel.

[0092] Accordingly, by performing channel-specific processing on any pixel, the luminance channel representation and color channel representation corresponding to that pixel will be obtained. Then, by combining the luminance channel representation and color channel representation corresponding to each pixel, the luminance channel image and color channel image corresponding to that pixel will be obtained, thereby indirectly realizing channel-specific processing on any HDR image. The purpose of using channel-specific processing in this application embodiment is to facilitate the subsequent separation of luminance tone mapping and color correction based on the luminance channel image and color channel image for any HDR image.

[0093] The second processing unit 13 is used to process the luminance channel image corresponding to the HDR image using a preset tone mapping algorithm to obtain the first image.

[0094] The third processing unit 14 is used to process the color channel image corresponding to the HDR image using a preset color correction rule to obtain the second image.

[0095] In the embodiments of this application, it should be noted that the terms "first image" and "second image" are used only to distinguish the images obtained by "brightness tone mapping" and "color correction" respectively, and do not have any other meaning of sequential reference.

[0096] The preset tone mapping algorithm can be implemented using the following tone mapping formula, such as formula (1): Output tone = tone mapping function (input tone);

[0097] The tone mapping function can be linear or non-linear; a linear mapping is used as an example here. Output tone = tone gain (chroma mapping index) * input tone. Using a preset tone gain curve, the tone mapping index is used to find the corresponding tone gain multiplied by the input tone to complete the tone mapping, thus obtaining the output tone. Based on the output tone, the HDR image is mapped onto a display device to output the displayed image. It should be noted that this embodiment performs tone mapping processing on the luminance channel image, and the tone mapping index is mainly constructed based on luminance characteristics. The specific construction method is not limited in this embodiment.

[0098] Among them, the preset color correction rules refer to the correction rules pre-defined for different color features. In this embodiment of the application, it should be emphasized that the terms "color features" and "color characteristics" are different. Color features refer to color features other than brightness features, such as saturation features, hue features, or custom features, etc. Based on the three attributes of "color"—"brightness," "saturation," and "hue"—"color features" can refer to brightness features, saturation features, or hue features. Therefore, the technical solution of this application uses "color features" to emphasize that it is only for color processing.

[0099] In this embodiment of the application, after the color feature is selected, the feature value of the color feature corresponding to each pixel can be calculated for the original HDR image corresponding to a color channel image to be processed. For example, its specific implementation method includes, but is not limited to: normalizing the RGB value of each pixel based on the selected color feature to obtain the feature value of the preset color feature corresponding to each pixel.

[0100] For example, taking saturation features as an example, the following saturation feature normalization formula is used, as shown in formula (2):

[0101]

[0102] Wherein, “saturation” refers to “saturation feature value”; Max(R, G, B) refers to selecting the maximum component value from the red, green and blue color channel components of a pixel; Min(R, G, B) refers to selecting the minimum component value from the red, green and blue color channel components of a pixel. For this formula (2), the saturation feature value corresponding to each pixel after normalization is in the range of 0 to 1.

[0103] Furthermore, this application embodiment takes saturation features as an example to illustrate the specific implementation process of obtaining a second image by processing a color channel image using preset color correction rules, as explained below:

[0104] Preset color correction rules based on saturation features, for example, are as follows: Figure 2 The saturation correction gain curve shown is as follows. Figure 2 The horizontal axis represents the saturation feature value corresponding to each pixel, and the vertical axis represents the gain correction parameter. Figure 2 The curve shown is achieved using the following saturation correction formula, as shown in formula (3):

[0105] u'=u*scale, v'=v*scale formula (3);

[0106] Where u and v are the color components of the YUV color space of the input image; u' and v' are the color components of the corrected YUV color space; and scale is the gain correction parameter, which is referenced from... Figure 2 The curve shown is illustrated. In this embodiment, color correction can be performed on each pixel in the color channel image based on the color components of the corrected YUV color space.

[0107] The output unit 15 is used to output the target display screen corresponding to each HDR image based on the first image and the second image.

[0108] In this embodiment, for any HDR image, since the luminance mapping and color correction are performed separately for the luminance channel image and the color channel image, when the HDR image is output and displayed using a display device, it is also necessary to fuse the results of the separate processing. That is, the first image and the second image are fused based on the same HDR image, which is equivalent to obtaining a target image that has undergone luminance and hue mapping and color correction at the same time as the HDR image. The target display screen is output using the target image by the display device, thereby ensuring that the hue mapping quality and color are not distorted at the same time, and providing a better picture quality viewing experience.

[0109] This application provides a high dynamic range image processing apparatus based on tone mapping. The apparatus includes an acquisition unit 11, a first processing unit 12, a second processing unit 13, a third processing unit 14, and an output unit 15. This application obtains multiple consecutive frame HDR images by acquiring HDR video data using the acquisition unit 11. For each HDR image, the first processing unit 12 performs channel-specific processing to obtain a luminance channel image and a color channel image. The second processing unit 13 then performs tone mapping processing based on the luminance channel image to obtain a first image, and the third processing unit 14 performs color correction processing based on the color channel image to obtain a second image. Therefore, this application provides a processing scheme for HDR images by separating luminance tone mapping and color correction, and then outputting a target display image based on the processing results using a display device. Compared to existing technologies, this invention solves the technical problem that compressing only brightness when performing tone mapping on HDR images leads to color distortion and low image quality. The brightness and color channel processing scheme provided in this application is equivalent to performing color correction while performing brightness tone mapping on HDR images. This ensures that tone mapping does not result in image quality loss and that color is not distorted, thereby improving the image quality of the images output by the display device.

[0110] In some modified embodiments, this application also provides another high dynamic range image processing apparatus based on tone mapping, such as... Figure 3 As shown, the apparatus provided in the above embodiments is further refined to supplement more functional applications.

[0111] like Figure 3 As shown, the first processing unit 12 is used to perform channel-by-channel processing on each HDR image to obtain the luminance channel image and color channel image corresponding to each HDR image. The first processing unit 12 is further divided into:

[0112] The first acquisition module 121 is used to acquire multiple pixels contained in the HDR image; the parsing module 122 is used to parse the luminance component value and color component value from the pixels; the first construction module 123 is used to construct the luminance channel image corresponding to the HDR image based on the luminance component value corresponding to each pixel; the first construction module 123 is also used to construct the color channel image corresponding to the HDR image based on the color component value corresponding to each pixel.

[0113] like Figure 3 As shown in the embodiments of this application, another high dynamic range image processing apparatus based on tone mapping further includes:

[0114] The partitioning unit 16 is used to divide multiple consecutive HDR images into multiple image frame groups after performing channel-based processing on each HDR image to obtain the corresponding luminance channel image and color channel image of each HDR image. Each image frame group contains multiple adjacent HDR images.

[0115] The construction unit 17 is used to construct a first channel image group and a second channel image group corresponding to each image frame group, wherein the first channel image group contains a luminance channel image and the second channel image group contains a color channel image.

[0116] In the embodiments of this application, the multiple consecutive HDR images contained in the acquired HDR video data can be divided into multiple groups of image frames. Specifically, the division method can be, but is not limited to, equal division, unequal division, or a combination of both.

[0117] It should be noted that the embodiments of this application aim to divide the images represented by the original HDR video data into multiple small and different small instance scenes by grouping them. The consecutive frame images contained in each small instance scene present similar color effects, but the color effects presented by different small instance scenes are different. This makes it convenient to use different color correction gains in each small instance scene, while using the same correction gain in a small actual scene. This way, it is not necessary to perform the operation of obtaining the required color correction gain for each image, thereby reducing processing costs while still ensuring that the expected effect of color correction is achieved.

[0118] Accordingly, embodiments of this application may, but are not limited to, divide these consecutive HDR frames into groups based on the color similarity between each HDR image, obtaining a small instance scene corresponding to each image frame group. Furthermore, based on the grouping of HDR image frames, and combined with the luminance channel image and color channel image obtained from the channel-specific processing of each HDR image, a luminance channel image group and a color channel image group corresponding to each image frame group are constructed. It should be noted that, for ease of distinguishing the channel-specific processing results of the image frame groups, the luminance channel image group is labeled as the "first channel image group," and the color channel image group is labeled as the "second channel image group."

[0119] like Figure 3 As shown, the building unit 17 can be further divided into:

[0120] The second construction module 171 is used to construct a first channel image group corresponding to the image frame group based on the brightness channel image corresponding to each HDR image in the image frame group; the second construction module 171 is also used to construct a second channel image group corresponding to the image frame group based on the color channel image corresponding to each HDR image in the image frame group; the association module 172 is used to establish a mapping relationship between the first channel image group and the second channel image group based on the same image frame group.

[0121] In this embodiment, any HDR image is processed by channel division to obtain a luminance channel image and a color channel image. These two images actually belong to the same HDR image. Based on this attribution relationship, after channel division processing of each HDR image within an image frame group, the first image group and the second image group corresponding to that image frame group have a mapping relationship. Furthermore, based on the same HDR image, there is also a correlation between the images within the first image group and the images within the second image group.

[0122] like Figure 3 As shown, the third processing unit 14 can be further divided into:

[0123] The second acquisition module 141 is used to acquire the target image that is centered in the second channel image group, wherein the ranking of each image in the second channel image group is the same as the ranking of the corresponding HDR image in the image frame group; the determination module 142 is used to use the target image as the feature image corresponding to the second channel image group; the processing module 143 is used to process the feature image using a preset color correction rule to obtain the corresponding color channel correction gain value; the correction module 144 is used to correct each color channel image in the second channel image group according to the color channel correction gain value to obtain the second image corresponding to each color channel image.

[0124] In this embodiment, since multiple consecutive HDR images have been divided into groups and a second image group (i.e., color channel image group) corresponding to each image frame group has been obtained, the third processing unit 14 is used to perform color correction on each color channel image group.

[0125] In a preferred manner, during the color correction process for each image in the second image group, since each image frame group corresponds to a small instance scene and the colors of the HDR images contained therein are similar, the colors of each image in the second image group are also similar. Therefore, in this embodiment, one or more images can be selected as feature images, and the color correction gain obtained from the feature images can be used to perform color correction on the entire second image group. This avoids the need to obtain the color correction gain of each image in the second image group in advance, which would result in too much unnecessary processing cost.

[0126] For example, since the ranking of each image in the second channel image group is the same as the ranking of the corresponding HDR image in the image frame group, the embodiments of this application preferentially obtain the target image with the middle ranking from the second channel image group as the feature image, thereby ensuring that the color correction gain obtained for the feature image is applicable to the entire second image group.

[0127] like Figure 3 As shown, the processing module 143 can be further divided into:

[0128] The first acquisition submodule 1431 is used to acquire multiple pixels contained in the feature image; the calculation submodule 1432 is used to calculate the feature value of the preset color feature corresponding to each pixel based on the preset color feature; the first determination submodule 1433 is used to determine the hue gain value corresponding to the feature value by finding the preset gain curve corresponding to the preset color feature; and the composition submodule 1434 is used to compose the color channel correction gain value corresponding to the feature image based on the preset color feature and the hue gain value corresponding to the feature value of each pixel.

[0129] In this embodiment, color features refer to color features other than brightness features, such as saturation features and hue features. This embodiment can pre-set preset color correction rules for each color feature. The saturation feature is used as an example for explanation:

[0130] like Figure 2The example given is a preset gain curve (i.e., a saturation correction gain curve) corresponding to the saturation feature. For instance, in this embodiment, the saturation value corresponding to each pixel is normalized and converted into a value between 0 and 1, which is used as the saturation feature value corresponding to each pixel. Based on the different values ​​of the saturation feature value on the horizontal axis, different "gain correction parameters" are selected using such a preset gain curve to obtain the "gain correction parameters" corresponding to each pixel. The data set composed of these "gain correction parameters" is used as the color channel correction gain value corresponding to the entire image.

[0131] like Figure 3 As shown, the calculation submodule 1432 can be further divided into the following:

[0132] The second acquisition submodule 14321 is used to acquire the RGB values ​​corresponding to each pixel; the normalization processing submodule 14322 is used to normalize the RGB values ​​of each pixel based on the preset color features to obtain the feature values ​​of the preset color features corresponding to each pixel.

[0133] In a preferred embodiment, this application uses a normalization process to obtain the feature value corresponding to each pixel on a specified color feature. Taking the saturation feature value as an example, since its value is between 0 and 1, it is convenient to measure the level of saturation.

[0134] For example, saturation levels below 0.1 are defined as low saturation, above 0.9 as high saturation, and in between as medium saturation. This utilizes preset color correction rules corresponding to saturation characteristics to achieve the following: high saturation areas are prone to distortion exceeding the color gamut due to brightness mapping, so saturation correction is applied to reduce it; medium saturation areas are prone to saturation decrease due to brightness mapping, so saturation needs to be increased; while low saturation areas can maintain their original saturation.

[0135] like Figure 3 As shown, if there are multiple preset color features, the component submodule 1434 can be further divided into the following:

[0136] The allocation submodule 14341 is used to assign weights to multiple preset color features; the second determination submodule 14342 is used to determine the tone gain value corresponding to the preset color feature based on the feature value of the preset color feature corresponding to each pixel; and the construction submodule 14343 is used to construct the color channel correction gain value corresponding to the feature image based on the weights corresponding to the multiple preset color features and the tone gain value corresponding to the preset color feature.

[0137] In this embodiment, color correction processing of a color channel image can be performed based on one or more color features. It should be noted that if multiple color features are used for the operation, a preferred approach is to assign weights to different color features, configuring the proportion of the hue gain value corresponding to each color feature in the calculation based on these weights. This effectively integrates multiple color features to achieve a higher quality color correction operation for the color channel image.

[0138] Furthermore, as a response to the above Figure 1 The application of the device shown in this application provides a method for processing high dynamic range images based on tone mapping. This method embodiment corresponds to the foregoing device embodiment. For ease of reading, this method embodiment will not repeat the details of the foregoing device embodiment, but it should be understood that the method in this embodiment can implement all the contents of the foregoing device embodiment. Figure 4 As shown, the following specific steps are provided in this embodiment of the application:

[0139] 201. Obtain HDR video data, which contains multiple consecutive HDR images.

[0140] 202. Perform channel-specific processing on each HDR image to obtain the corresponding luminance channel image and color channel image for each HDR image.

[0141] 203a. Process the luminance channel image corresponding to the HDR image using a preset tone mapping algorithm to obtain the first image.

[0142] 203b. Process the color channel image corresponding to the HDR image using preset color correction rules to obtain the second image.

[0143] 204. Based on the first image and the second image, output the target display screen corresponding to each HDR image.

[0144] Furthermore, as a response to the above Figure 3 The application of the device shown in this application provides another method for processing high dynamic range images based on tone mapping. This method embodiment corresponds to the foregoing device embodiment. For ease of reading, this method embodiment will not repeat the details of the foregoing device embodiment, but it should be understood that the method in this embodiment can implement all the contents of the foregoing device embodiment. Figure 5 As shown, the following specific steps are provided in this embodiment of the application:

[0145] 301. Obtain HDR video data, which contains multiple consecutive HDR images.

[0146] 302. Perform channel-specific processing on each HDR image to obtain the corresponding luminance channel image and color channel image for each HDR image.

[0147] In this embodiment of the application, this step can be further implemented as follows: First, obtain multiple pixels contained in the HDR image; second, parse the luminance component value and color component value from the pixels; finally, construct the luminance channel image corresponding to the HDR image based on the luminance component value corresponding to each pixel, and construct the color channel image corresponding to the HDR image based on the color component value corresponding to each pixel.

[0148] 303. Divide multiple consecutive HDR images into multiple image frame groups, with each image frame group containing multiple adjacent HDR images.

[0149] 304. Construct a first channel image group and a second channel image group corresponding to each image frame group, wherein the first channel image group contains a luminance channel image and the second channel image group contains a color channel image.

[0150] In the embodiments of this application, this step can be further implemented as follows: First, based on the luminance channel image corresponding to each HDR image in the image frame group, a first channel image group corresponding to the image frame group is constructed; second, based on the color channel image corresponding to each HDR image in the image frame group, a second channel image group corresponding to the image frame group is constructed; and third, based on the same image frame group, a mapping relationship is established between the first channel image group and the second channel image group.

[0151] 305a. Process the luminance channel image corresponding to the HDR image using a preset tone mapping algorithm to obtain the first image.

[0152] In this embodiment, steps 305b-308b are used to process the color channel image corresponding to the HDR image using a preset color correction rule to obtain a second image. The specific explanation is as follows:

[0153] 305b. Obtain the target image that is centered in the second channel image group, wherein the rank of each image in the second channel image group is the same as the rank of the corresponding HDR image in the image frame group.

[0154] 306b. Use the target image as the feature image corresponding to the second channel image group.

[0155] 307b. Process the feature image using preset color correction rules to obtain the corresponding color channel correction gain value.

[0156] In this embodiment of the application, this step can be further implemented as follows: First, obtain multiple pixels contained in the feature image; second, based on the preset color features, obtain the feature values ​​of the preset color features corresponding to each pixel; finally, by finding the preset gain curve corresponding to the preset color features, determine the hue gain value corresponding to the feature value, and based on the preset color features, form the color channel correction gain value corresponding to the feature image according to the hue gain value corresponding to the feature value corresponding to each pixel.

[0157] The preferred method for obtaining the feature values ​​of the preset color features corresponding to each pixel is as follows:

[0158] Obtain the RGB value corresponding to each pixel; based on the preset color features, normalize the RGB value of each pixel to obtain the feature value of the preset color features corresponding to each pixel.

[0159] In this embodiment, color correction processing can be performed on a color channel image based on one or more color features. For example, a specific implementation method for obtaining the color channel correction gain value corresponding to the feature image during a correction operation based on multiple color features may include the following:

[0160] First, weights are assigned to multiple preset color features; second, the tone gain value corresponding to the preset color feature is determined based on the feature value of the preset color feature corresponding to each pixel; finally, the color channel correction gain value corresponding to the feature image is formed based on the weights of the multiple preset color features and the tone gain value corresponding to the preset color features.

[0161] 308b. Correct each color channel image in the second channel image group according to the color channel correction gain value to obtain the second image corresponding to each color channel image.

[0162] 309. Based on the first image and the second image, output the target display screen corresponding to each HDR image.

[0163] In summary, the high dynamic range image processing apparatus and method based on tone mapping provided in this application embodiment utilizes the apparatus provided in this application embodiment to perform channel-specific processing on multiple consecutive frame HDR images contained in the acquired HDR video data to obtain luminance channel images and color channel images, thereby facilitating the subsequent separate processing of luminance tone mapping and color correction for HDR images. Furthermore, this application embodiment processes a large number of HDR images in a grouped manner. During the luminance tone mapping process, the same tone mapping index is used for processing each image frame group, and during the color correction process, the color correction gain required for the feature images in the color image group is applied to the entire color image group. This group-based processing scheme replaces the need to obtain the luminance tone mapping index for each luminance channel image, or the need to obtain the color correction gain for each color channel image, thereby saving significant processing costs. The solution provided in this application embodiment reduces processing costs while still ensuring the achievement of the expected color correction effect.

[0164] The high dynamic range image processing apparatus based on tone mapping provided in this application includes a processor and a memory. The aforementioned acquisition unit, first processing unit, second processing unit, third processing unit, and output unit are all stored in the memory as program units, and the processor executes the aforementioned program units stored in the memory to realize the corresponding functions.

[0165] The processor contains a kernel, which retrieves the corresponding program unit from memory. One or more kernels can be configured; by adjusting kernel parameters, color distortion after tone mapping of HDR images can be effectively corrected, improving the image quality of the output image from the display device.

[0166] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the high dynamic range image processing method based on tone mapping as described above.

[0167] This application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the high dynamic range image processing method based on tone mapping as described above.

[0168] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0169] In a typical configuration, the device includes one or more processors (CPUs), memory, and a bus. The device may also include input / output interfaces, network interfaces, etc.

[0170] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM, and memory includes at least one memory chip. Memory is an example of computer-readable media.

[0171] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0172] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0173] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0174] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A processing apparatus for high dynamic range images based on tone mapping, characterized in that, The device includes: The acquisition unit is used to acquire HDR video data, wherein the HDR video data contains multiple consecutive frame HDR images; The first processing unit is used to perform channel-based processing on each of the HDR images to obtain the luminance channel image and color channel image corresponding to each of the HDR images. The partitioning unit is used to divide multiple consecutive HDR images into multiple image frame groups after performing channel-splitting processing on each of the HDR images to obtain the luminance channel image and color channel image corresponding to each of the HDR images. Each image frame group contains multiple adjacent HDR images. A construction unit is used to construct a first channel image group and a second channel image group corresponding to each image frame group, wherein the first channel image group contains a luminance channel image and the second channel image group contains a color channel image; wherein, based on the same image frame group, there is a mapping relationship between the first channel image group and the second channel image group; each image frame group corresponds to a unique instance scene, and the color effects of different instance scenes are different; The second processing unit is used to process the luminance channel image corresponding to the HDR image using a preset tone mapping algorithm to obtain the first image; The third processing unit is used to process the color channel image corresponding to the HDR image using a preset color correction rule to obtain a second image; including: correcting different instance scenes using different color correction gains, and correcting each HDR image contained in the same instance scene using the same color correction gain; The output unit is configured to fuse the first image and the second image to obtain a target image after tone mapping and color correction, so as to output the target display screen corresponding to each HDR image using the target image corresponding to each HDR image.

2. The apparatus according to claim 1, characterized in that, The first processing unit includes: The first acquisition module is used to acquire multiple pixels contained in the HDR image; The parsing module is used to parse the luminance component value and the color component value from the pixel. The first construction module is used to construct the luminance channel image corresponding to the HDR image based on the luminance component values ​​corresponding to each pixel. The first construction module is further configured to construct a color channel image corresponding to the HDR image based on the color component values ​​corresponding to each pixel.

3. The apparatus according to claim 1, characterized in that, The building unit includes: The second construction module is used to construct a first channel image group corresponding to the image frame group based on the brightness channel image corresponding to each of the HDR images in the image frame group. The second construction module is further configured to construct a second channel image group corresponding to the image frame group based on the color channel images corresponding to each of the HDR images in the image frame group. The association module is used to establish a mapping relationship between the first channel image group and the second channel image group based on the same image frame group.

4. The apparatus according to claim 1, characterized in that, The third processing unit includes: The second acquisition module is used to acquire the target image that is in the center position from the second channel image group, wherein the position of each image in the second channel image group is the same as the position of the corresponding HDR image in the image frame group; The determining module is used to use the target image as the feature image corresponding to the second channel image group; The processing module is used to process the feature image using the preset color correction rules to obtain the corresponding color channel correction gain value; The correction module is used to correct each color channel image in the second channel image group according to the color channel correction gain value, so as to obtain the second image corresponding to each color channel image.

5. The apparatus according to claim 4, characterized in that, The processing module includes: The first acquisition submodule is used to acquire multiple pixels contained in the feature image; The calculation submodule is used to calculate the feature value of the preset color feature corresponding to each pixel point based on the preset color feature; The first determining submodule is used to determine the hue gain value corresponding to the feature value by finding the preset gain curve corresponding to the preset color feature; The submodule is used to compose the color channel correction gain value corresponding to the feature image based on the preset color features and the hue gain value corresponding to the feature value of each pixel.

6. The apparatus according to claim 5, characterized in that, The computational submodule includes: The second acquisition submodule is used to acquire the RGB value corresponding to each pixel. The normalization processing submodule is used to normalize the RGB values ​​of each pixel based on a preset color feature to obtain the feature value of the preset color feature corresponding to each pixel.

7. The apparatus according to claim 5, characterized in that, If there are multiple preset color features, then the sub-modules include: The allocation submodule is used to assign weights to multiple preset color features; The second determining submodule determines the hue gain value corresponding to the preset color feature based on the feature value of the preset color feature corresponding to each pixel. A submodule is constructed to construct the color channel correction gain value corresponding to the feature image based on the weights corresponding to the multiple preset color features and the hue gain value corresponding to the preset color features.

8. A method for processing high dynamic range images based on tone mapping, characterized in that, The method includes: Acquire HDR video data, which contains multiple consecutive HDR images; Each HDR image is processed by channel segmentation to obtain the luminance channel image and color channel image corresponding to each HDR image; Multiple consecutive HDR images are divided into multiple image frame groups, and each image frame group contains multiple adjacent HDR images; Construct a first-channel image group and a second-channel image group corresponding to each image frame group, wherein the first-channel image group contains a luminance channel image and the second-channel image group contains a color channel image; wherein, based on the same image frame group, there is a mapping relationship between the first-channel image group and the second-channel image group; each image frame group corresponds to a unique instance scene, and the color effects of different instance scenes are different; The luminance channel image corresponding to the HDR image is processed using a preset tone mapping algorithm to obtain a first image; The second image is obtained by processing the color channel image corresponding to the HDR image using a preset color correction rule; including: correcting different instance scenes using different color correction gains, and correcting each HDR image contained in the same instance scene using the same color correction gain; The first image and the second image are fused to obtain a target image that has undergone tone mapping and color correction, so as to output the target display screen corresponding to each HDR image using the target image corresponding to each HDR image.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the high dynamic range image processing method based on tone mapping as described in claim 8.

10. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the high dynamic range image processing method based on tone mapping as described in claim 8.

Images