Image tone mapping method and device, computer readable medium and electronic device

Abstract

The present disclosure provides an image tone mapping method and device, a computer-readable medium and an electronic device, and relates to the field of image processing technology. The method includes: obtaining an original image, and determining an initial color gain obtained by a first tone mapping process of the original image; obtaining spectral perception data corresponding to the original image; determining a target color gain based on the spectral perception data and the initial color gain; and performing a second tone mapping process on the original image according to the target color gain to obtain a target image. The present disclosure can enhance the color information of the image signal processing process corresponding to the Bayer sensor through the separately collected spectral perception data, improve the accuracy of the tone mapping result, ensure the color performance of the target image, and realize the color restoration and enhancement of the original image collected by the Bayer sensor.

Description

Background Art

[0002] With the continuous improvement of people's living standards, the quality of captured images has received more and more attention. Tone mapping refers to a computer graphics technology that approximates the display of high dynamic range images on a limited dynamic range medium. The main problem that tone mapping solves is to perform a large contrast attenuation to transform the scene brightness to a displayable range while maintaining image details and colors, which are very important information for representing the original scene.

[0003] At present, in the related schemes, the tone mapping process mostly processes the image based on the brightness data adjustment method. However, this scheme lacks compensation for the color loss or distortion after image processing, resulting in poor accuracy of image color restoration or image color enhancement and poor output image quality. Summary of the invention

[0004] The purpose of the present disclosure is to provide an image tone mapping method, an image tone mapping device, a computer-readable medium and an electronic device, thereby improving the accuracy of color restoration or color enhancement of an image after tone mapping processing, at least to a certain extent, and improving the quality of the output image.

[0005] According to a first aspect of the present disclosure, there is provided an image tone mapping method, comprising:

[0006] Acquire an original image, and determine an initial color gain of the original image obtained by a first tone mapping process;

[0007] Acquire spectral perception data corresponding to the original image;

[0008] Determining a target color gain based on the spectral perception data and the initial color gain;

[0009] A second tone mapping process is performed on the original image according to the target color gain to obtain a target image.

[0010] According to a second aspect of the present disclosure, there is provided an image tone mapping device, comprising:

[0011] An initial color gain determination module, used to obtain an original image and determine an initial color gain obtained by processing the original image through a first tone mapping process;

[0012] A spectral perception data acquisition module, used to acquire spectral perception data corresponding to the original image;

[0013] A target color gain determination module, configured to determine a target color gain based on the spectral perception data and the initial color gain;

[0014] The image tone mapping module is used to perform a second tone mapping process on the original image according to the target color gain to obtain a target image.

[0015] According to a third aspect of the present disclosure, a computer-readable medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented.

[0016] According to a fourth aspect of the present disclosure, there is provided an electronic device, characterized in that it includes:

[0017] Processor; and

[0018] The memory is used to store one or more programs. When the one or more programs are executed by one or more processors, the one or more processors implement the above method.

[0019] An image tone mapping method provided by an embodiment of the present disclosure can determine the initial color gain obtained by the original image after the first tone mapping process, obtain the spectral perception data corresponding to the original image, and then determine the target color gain based on the spectral perception data and the initial color gain. Finally, the original image can be subjected to a second tone mapping process according to the target color gain to obtain the target image. The accurate color data corresponding to the original image is obtained through the spectral perception data, and the target color gain finally applied to the original image is determined in combination with the initial color gain. While realizing the tone mapping of the image, the accuracy of the color restoration of the original image can be effectively improved, and the problem of image color loss or distortion caused by tone mapping based on brightness adjustment can be avoided, thereby improving the color restoration effect of the original image and ensuring the image quality of the target image.

[0020] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0022] Figure 1 A schematic diagram schematically illustrates a flow chart of an image tone mapping method in an exemplary embodiment of the present disclosure;

[0023] Figure 2A schematic diagram schematically illustrates a system architecture of an image processing system that can execute an image tone mapping method in an exemplary embodiment of the present disclosure;

[0024] Figure 3 A schematic diagram schematically illustrates a configuration of a spectrum sensing device and an electronic device in an exemplary embodiment of the present disclosure;

[0025] Figure 4 A schematic diagram schematically shows the principle of collecting spectral color data by a spectral sensor array in an exemplary embodiment of the present disclosure;

[0026] Figure 5 A schematic diagram of a process for determining a target global color gain in an exemplary embodiment of the present disclosure is schematically shown;

[0027] Figure 6 A schematic diagram of a process for determining a target local color gain in an exemplary embodiment of the present disclosure is schematically shown;

[0028] Figure 7 A schematic diagram schematically illustrates a difference weight curve in an exemplary embodiment of the present disclosure;

[0029] Figure 8 A schematic diagram schematically illustrates a principle of calculating a target local color gain in an exemplary embodiment of the present disclosure;

[0030] Fig. 9 A schematic diagram schematically shows the composition of an image tone mapping device in an exemplary embodiment of the present disclosure;

[0031] Fig.10 A schematic diagram of an electronic device to which the embodiments of the present disclosure can be applied is shown. DETAILED DESCRIPTION

[0032] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0033] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0034] In the related art, tone mapping technology is processed based on brightness. It is usually divided into global tone mapping (GTM) and local tone mapping (LTM). Global tone mapping has a fast calculation speed, but it cannot distinguish and process according to information content. Local tone mapping can adjust the mapping function according to the local information of the space.

[0035] Current tone mapping technology solutions are based on brightness adjustment, and rarely combine color change information for further processing. Since brightness and color together constitute the color space, most tone mapping processes tend to only focus on brightness adjustment solutions, lacking compensation for color loss or distortion after processing, resulting in poor accuracy in color restoration or color enhancement of tone mapped images, and poor image quality.

[0036] Based on one or more problems in the related art, the present disclosure first provides an image tone mapping method. The image tone mapping method of an exemplary embodiment of the present disclosure is specifically described below by taking an electronic device executing the method as an example.

[0037] Figure 1 A schematic flow chart of an image tone mapping method in this exemplary embodiment is shown, which may include the following steps S110 to S140:

[0038] In step S110, an original image is acquired, and an initial color gain of the original image obtained by a first tone mapping process is determined.

[0039] In an exemplary embodiment, the original image refers to an image generated by an image sensor collecting light signals and has not yet been processed by an image signal processor (Image Signal Processing, ISP).

[0040] The first tone mapping process refers to the process of performing tone mapping on the original image based on the brightness value in the original image. For example, in the relevant image signal processing process, tone mapping modules can be set in different processing modules, such as setting tone mapping modules in the Raw RGB domain processing module, the Full RGB domain processing module, and the YUV domain processing module. Each tone mapping module may include a global tone mapping module and a local tone mapping module. Each tone mapping module may perform global tone mapping and local tone mapping on the image data in different image domain processing modules, and finally obtain the initial color gain of the original image in the different image domain processing modules. The initial color gain may include a global color gain and a local color gain.

[0041] Among them, global tone mapping refers to the statistics of the entire original image and the pixel-by-pixel mapping of the entire image information using the same mapping function according to the histogram information of the entire image. After mapping the input pixels, the output pixels are obtained, and the color gain of each pixel is obtained by using the ratio of the output pixel to the input pixel, and finally the global color gain of the original image is obtained.

[0042] Local color mapping refers to dividing the entire original image into blocks, and statistically analyzing the pixel information of each block in the area of ​​interest, calculating the mapping function in each block, and mapping the input pixels in each block to obtain the output pixels. The ratio of the output pixels to the input pixels is used to determine the color gain of each block, and finally the local color gain in each block of the original image is obtained.

[0043] In step S120, spectral perception data corresponding to the original image is obtained.

[0044] In an exemplary embodiment, spectral perception data refers to data obtained by perceiving the real colors in the shooting scene through a spectral perception device when shooting the original image. For example, the spectral perception device can be a multi-spectral sensor or a CCD direct-reading spectrometer. The spectral perception data in the shooting scene corresponding to the original image is obtained through the spectral perception device. Compared with the color data obtained by the image sensor based on the RGB three channels, the spectral perception data can include the spectral colors of visible light in a wider range, the color information is richer, and the color information in the real shooting scene is better restored.

[0045] When the image sensor acquires the original image, the spectrum sensing device can be started to acquire the spectrum sensing data in the shooting scene corresponding to the original image, and the original image and the spectrum sensing data are used as input data of the image signal processor.

[0046] In step S130 , a target color gain is determined based on the spectral perception data and the initial color gain.

[0047] In an exemplary embodiment, the target color gain refers to a new color gain obtained by fusing the spectral perception data and the initial color gain. For example, the initial color gain can be applied to the original image to obtain a new pixel value. The new pixel value can be compared with the spectral perception data to determine the most appropriate contrast value at the current position of the original image. Assuming that the most appropriate contrast value can be the spectral perception data, a new color gain can be determined based on the spectral perception data, and the initial color gain at the position can be replaced by the new color gain to obtain the target color gain. Of course, the region of interest in the original image (such as a portrait region) can also be determined, and the new color gain corresponding to the region of interest can be determined based on the spectral perception data corresponding to the region of interest. The initial color gain is still used for other original image regions to obtain the target color gain. This exemplary embodiment does not impose any special limitation on the method of fusing the spectral perception data and the initial color gain to obtain the target color gain result.

[0048] Optionally, the spectral perception data and the initial color gain can be divided into regions, and the spectral perception data and the initial color gain in the same region can be compared. According to the difference between the two, the target color gain can be obtained by using the spectral perception data, the initial color gain, or a weighted fusion of the spectral perception data and the initial color gain.

[0049] In step S140, a second tone mapping process is performed on the original image according to the target color gain to obtain a target image.

[0050] In an exemplary embodiment, the second tone mapping process refers to applying the target color gain to the original image to achieve tone mapping of the original image. By incorporating spectral perception data into the tone mapping process, compared with tone mapping based on brightness adjustment in related schemes, more color information in the shooting scene is considered, and the problem of image color loss or distortion caused by tone mapping based on brightness adjustment is avoided, the accuracy of color restoration of the image after color mapping is effectively guaranteed, and the image quality of the target image is effectively improved.

[0051] Steps S110 to S140 are described in detail below.

[0052] In an example embodiment, the image tone mapping method may be performed by an electronic device provided with an image processing system. Specifically, the image processing system may include at least one set of spectral sensing devices and at least one set of camera modules. Figure 2 A schematic diagram of the system architecture of an image processing system that can execute an image tone mapping method in an exemplary embodiment of the present disclosure is schematically shown.

[0053] refer to Figure 2As shown, the image processing system 200 may include: a spectral sensing device 210 can be used to obtain spectral sensing data in a shooting scene; the camera module 220 may include an image sensor 221 and an image signal processor 222, wherein: the image sensor 221 can be used to obtain an original image; the image signal processor 222 can be electrically connected to the image sensor 221 and the spectral sensing device 210, and can be used to determine an initial color gain of the original image obtained by a first tone mapping process, and based on the spectral sensing data and the initial color gain, determine a target color gain, and finally perform a second tone mapping process on the original image according to the target color gain to obtain a target image.

[0054] Among them, the spectral sensing device 210 refers to a sensing unit for sensing the real color in the shooting scene when shooting the original image. For example, the spectral sensing device 210 can be a multispectral sensor (Multispectral sensor), which can include imaging optical elements and optical elements for dividing the spectrum, and can accurately identify all spectral color information in the current scene; of course, the spectral sensing device 210 can also be a CCD direct-reading spectrometer, and this example embodiment does not specifically limit the type of spectral sensing device. Spectral color data refers to the color data in the real scene that the spectral sensing device 210 synchronously collects when the camera module 220 shoots the original image.

[0055] The image sensor 221 refers to a sensor that collects light signals based on a Bayer filter pattern template and converts the light signals into electrical signals. For example, the image sensor 221 can be a complementary metal oxide semiconductor CMOS or a charge coupled detection device CCD. This example embodiment does not impose any special limitation on the sensor type of the image sensor 221.

[0056] The image processing system 200 may be provided in an electronic device having certain computing capabilities. For example, the electronic device may be a desktop computer, a portable computer, a smart phone, an intelligent robot, a wearable device, a tablet computer, etc. This example embodiment does not specifically limit this.

[0057] In an exemplary embodiment, spectral perception data corresponding to the original image can be acquired through a preset spectral perception device, wherein the spectral perception device can include a spectral sensor array composed of at least two spectral sensors, and the spectral sensor array can generate spectral perception data corresponding to at least two detection areas.

[0058] Figure 3 A schematic diagram schematically illustrates a configuration of a spectrum sensing device and an electronic device in an exemplary embodiment of the present disclosure.

[0059] refer to Figure 3As shown, taking the spectral sensing device 210 as a multispectral sensor 310 as an example, the multispectral sensor 310 can be integrated into the camera module 220, or can be set in the electronic device independently of the camera module 220. For example, it can be set on the housing 320 of the electronic device, and the distance from the camera module 220 is within the distance threshold; of course, the spectral sensing device 210 can also be used as an external device to communicate with the electronic device corresponding to the camera module 220. For example, the external spectral sensing device 330 can communicate with the camera module 220 in a wired or wireless manner 340. This example embodiment does not specifically limit the connection method of the spectral sensing device 210.

[0060] Figure 4 The schematic diagram schematically shows the principle of collecting spectral color data by a spectral sensor array in an exemplary embodiment of the present disclosure.

[0061] refer to Figure 4 As shown, the spectrum sensing device may include multiple spectrum sensors 410, and the spectrum sensors 410 are arranged in space according to a preset number of columns and rows to obtain a spectrum sensor array. Specifically, the multiple spectrum sensors 410 can be arranged in an M*N array to form a spectrum sensor array 420. For example, M can be preset to 10, and N can be set to 10. Then the spectrum sensing device may include 100 spectrum sensors 410, and these 100 spectrum sensors 410 are arranged in space to obtain a 10*10 spectrum sensor array. Of course, the specific M and N can be customized according to the actual application scenario, and this exemplary embodiment does not specifically limit this.

[0062] When shooting the original image, the spectral sensing device having a spatial structure of an M*N spectral sensor array 420 collects real color information in the shooting scene and obtains spectral color data 440 having M*N detection areas 430. The spectral color data in each detection area 430 can collect spectral color data in the corresponding real scene area. Compared with image sensors based on RGB channels, richer color information in the real scene can be obtained, a larger range of spectral information can be perceived, and the color information of the image sensor can be effectively supplemented. Compared with the method of only adjusting brightness in the related art, the problem of color loss or distortion after tone mapping can be effectively avoided, and the accuracy of color restoration of the target image after tone mapping can be improved.

[0063] In an exemplary embodiment, the target color gain may include a target global color gain, which may be obtained by Figure 5 The steps in the above are to determine the target color gain based on the spectral perception data and the initial color gain, refer to Figure 5 As shown, it may specifically include:

[0064] Step S510, performing global tone mapping on the original image to determine an initial global color gain;

[0065] Step S520, comparing the initial global color gain with the spectral perception data to obtain a comparison result;

[0066] Step S530: determining a target global color gain according to the comparison result.

[0067] Among them, global tone mapping performs statistics on the entire original image and uses the same mapping function to perform pixel-by-pixel mapping on the entire image information according to the histogram information of the entire image. After mapping the input pixels, the output pixels are obtained, and the color gain of each pixel is obtained by using the ratio of the output pixel to the input pixel, and finally the global color gain of the original image is obtained.

[0068] The initial global color gain may be compared with the spectral perception data to obtain a comparison result, and then the target global color gain may be determined according to the comparison result.

[0069] Optionally, the target global color gain can be determined based on the comparison result by the following steps: a preset first difference threshold and a preset second difference threshold can be obtained, the initial global color gain is applied to the original image to obtain global color data, and the difference data between the global color data and the spectral perception data is determined; if it is determined that the difference data is less than or equal to the first difference threshold, the initial global color gain is used as the target global color gain.

[0070] Among them, the difference data refers to the difference between the global color data and the spectral perception data obtained by applying the initial global color gain to the original image. For example, the initial global color gain can be applied to the original image to obtain the global color data, and the global color data of each pixel is subtracted from the spectral perception data of the corresponding pixel to obtain the difference data of each pixel; of course, the first average value of the global color data of all pixels in the original image can be calculated, the second average value of the spectral perception data of all pixels can be calculated, and the difference between the first average value and the second average value can be calculated as the difference data; this example embodiment does not specifically limit the method for calculating the difference data between the global color data and the spectral perception data, and can be customized according to actual usage.

[0071] If the difference data is the difference between the global color data and the spectral perception data at each pixel, the corresponding first difference threshold and the second difference threshold can be the thresholds for judging the difference data of a single pixel; if the difference data is the difference between the average values ​​of all pixels of the global color data and the spectral perception data, the corresponding first difference threshold and the second difference threshold can be the thresholds for judging the average values ​​of all pixels in the global color data and the spectral perception data corresponding to the original image. The specific difference threshold can be customized according to needs, and the present example embodiment is not limited to this.

[0072] When it is determined that the difference data is less than or equal to the first difference threshold, it can be considered that the initial global color gain obtained after the first tone mapping process is not much different from the color represented by the spectral perception data, and the initial global color gain is relatively accurate. Therefore, the initial global color gain can be directly used as the target global color gain of the original image.

[0073] Optionally, the target global color gain may be determined according to the comparison result by the following steps:

[0074] When it is determined that the difference data is greater than a first difference threshold and less than or equal to a second difference threshold, a difference weight curve can be obtained; then, based on the difference data and the difference weight curve, target weight data can be determined; finally, the global color data and the spectral perception data can be interpolated according to the target weight data to determine the target global color gain.

[0075] The difference weight curve refers to a preset curve used to determine weight data for interpolation calculation between global color data and spectral perception data. The target weight data refers to weight data determined in the difference weight curve based on the difference data.

[0076] When it is determined that the difference data is greater than the first difference threshold and less than or equal to the second difference threshold, the global color data and the spectral perception data may be interpolated according to the target weight data to determine the target global color gain of the original image. For example, when it is determined that the difference data is greater than the first difference threshold and less than or equal to the second difference threshold, the target global color gain may be calculated by equation (1):

[0077] P Gout =w*P G +(1-w)*P Color (1)

[0078] Among them, P Gout It can represent the target global color gain, P G Can represent global color data, P Colorcan represent spectral perception data, and w can represent target weight data, that is, weight data determined in a difference weight curve according to difference data between global color data and spectral perception data.

[0079] When it is determined that the difference data is greater than the first difference threshold and less than or equal to the second difference threshold, it can be considered that the initial global color gain is inaccurate, but the degree of deviation from the accurate value is low. In order to ensure the continuity and smoothness of the image color after the image global tone mapping, the target weight data can be determined in the difference weight curve according to the difference data, and then the global color data obtained by applying the initial global color gain to the original image and the spectral perception data are interpolated through the target weight data and the relationship (1), so as to determine the target global color gain of the original image.

[0080] Optionally, the target global color gain may be determined according to the comparison result by the following steps: when it is determined that the difference data is greater than a second difference threshold, the target global color gain may be calculated according to the spectral perception data.

[0081] When it is determined that the difference data is greater than the second difference threshold, it can be considered that the initial global color gain calculated by the first tone mapping processing of the original image is inaccurate, and the degree of inaccuracy is large. At this time, the global color gain can be abandoned, and then the target global color gain of the original image can be directly calculated according to the spectral perception data corresponding to the original image, so as to improve the accuracy of image color restoration of the original image after tone mapping.

[0082] In an exemplary embodiment, the target color gain may include a target local color gain, which may be obtained by Figure 6 The steps in the above are to determine the target color gain based on the spectral perception data and the initial color gain, refer to Figure 6 As shown, it may specifically include:

[0083] Step S610, determining a block area array corresponding to the original image, wherein the block areas in the block area array are consistent in size and position with the detection areas in the spectrum sensing device;

[0084] Step S620, counting pixel information corresponding to the block areas of the original image, and determining an initial local color gain of each of the block areas according to the pixel information;

[0085] Step S630, comparing the initial local color gain of each of the block areas with the spectral perception data of the detection area corresponding to the block area to obtain a comparison result;

[0086] Step S640: determining a target local color gain according to the comparison result.

[0087] Among them, the block area array corresponding to the original image can be determined, and the block areas in the block area array can be consistent with the size and position of the detection area in the spectral sensing device. Specifically, the block areas in the block area array can be consistent with the number, size, and position of the detection area in the spectral sensing device. Of course, the number of block areas in the block area array can also be greater than the number of detection areas in the spectral sensing device, and this example embodiment does not specifically limit this.

[0088] The target block area refers to the block area currently being processed among all the block areas of the original image, and the target detection area refers to the detection area in the spectral perception data corresponding to the target block area.

[0089] The initial local color gain refers to the result data obtained after local tone mapping processing is performed on the original image of each block area. Local color mapping refers to dividing the entire original image into blocks, and statistically analyzing the pixel information of each block area in the area of ​​interest, calculating the mapping function in each block area, and mapping the input pixels in each block area to obtain the output pixels. The color gain of each block area is determined by the ratio of the output pixel to the input pixel, and finally the local color gain in each block area of ​​the original image is obtained.

[0090] Optionally, the target local color gain may be determined according to the comparison result by the following steps:

[0091] A preset third difference threshold and a preset fourth difference threshold can be obtained, and the initial local color gain is applied to the original image to obtain local color data of each block area, and the difference data between the local color data and the spectral perception data is determined; when it is determined that the difference data is less than or equal to the first difference threshold, the initial local color gain can be used as the target local color gain.

[0092] The calculation method of the difference data is the same as the calculation method of the initial global color gain, except that the calculation data is converted from the global pixels of the original image to each block area in the original image, which will not be described in detail here.

[0093] The third difference threshold and the fourth difference threshold may be consistent with or inconsistent with the first difference threshold and the second difference threshold in the process of determining the target global color gain. The specific settings may be based on actual conditions and are not specifically limited here.

[0094] When it is determined that the difference data is less than or equal to the third difference threshold, it can be considered that the local color data of each block area obtained by applying the initial local color gain to the original image is not much different from the color represented by the spectral perception data, and the initial local color gain processed by the first tone mapping is relatively accurate. Therefore, the initial local color gain in the target block area can be directly used as the target local color gain of the target block area.

[0095] Optionally, the target local color gain may be determined according to the comparison result by the following steps:

[0096] When it is determined that the difference data is greater than the third difference threshold and less than or equal to the fourth difference threshold, a difference weight curve can be obtained, and then the target weight data can be determined based on the difference data and the difference weight curve; the local color data and the spectral perception data can be interpolated according to the target weight data to determine the target local color gain.

[0097] The difference weight curve refers to a pre-set curve for determining weight data for interpolation calculation between local color data and spectral perception data, and the difference weight curve and the curve for determining weight data for interpolation calculation between global color data and spectral perception data may be the same curve. The target weight data refers to the weight data determined in the difference weight curve based on the difference data.

[0098] When it is determined that the difference data is greater than the third difference threshold and less than or equal to the fourth difference threshold, the local color data and the spectral perception data may be interpolated according to the target weight data to determine the target local color gain of the original image. For example, when it is determined that the difference data is greater than the third difference threshold and less than or equal to the fourth difference threshold, the target local color gain may be calculated by equation (2):

[0099] P Lout =w*P L +(1-w)*P Color (2)

[0100] Among them, P Lout It can represent the target local color gain, P L Can represent local color data, P Color can represent spectral perception data, and w can represent target weight data, that is, weight data determined in a difference weight curve according to difference data between local color data and spectral perception data.

[0101] When it is determined that the difference data is greater than the third difference threshold and less than or equal to the fourth difference threshold, it can be considered that the initial local color gain is inaccurate, but the degree of deviation from the accurate value is low. In order to ensure the continuity and smoothness of the image color after the local tone mapping of the image, the target weight data can be determined in the difference weight curve according to the difference data, and then the local color data obtained by applying the initial local color gain to the original image and the spectral perception data are interpolated through the target weight data and the relationship (2), so as to determine the target local color gain of the original image.

[0102] Optionally, the target local color gain may be determined according to the comparison result by the following steps: when it is determined that the difference data is greater than a fourth difference threshold, the target local color gain may be calculated according to the spectral perception data.

[0103] When it is determined that the difference data is greater than the fourth difference threshold, it can be considered that the initial local color gain calculated by the first tone mapping processing of the original image is inaccurate, and the degree of inaccuracy is relatively large. At this time, the initial local color gain in the target block area can be abandoned, and then the spectral perception data in the target detection area can be directly used to calculate the target local color gain in the target block area to improve the accuracy of color restoration of the original image.

[0104] Figure 7 A schematic diagram schematically illustrates a difference weight curve in an exemplary embodiment of the present disclosure.

[0105] refer to Figure 7 As shown, the difference weight curve can be represented by the curve coordinate system in the figure, the vertical axis can represent the weight data 710, and the horizontal axis can represent the difference data 720 between the global (local) color data obtained by applying the initial global (local) color gain to the original image and the spectral perception data (of each detection area).

[0106] When it is determined that the difference data is less than or equal to the first (third) difference threshold 730, the weight data can be 1 at this time, and the initial global (local) color gain is directly used as the target global (local) color gain by substituting it into the relationship (1) or the relationship (2) for calculation. When it is determined that the difference data is greater than the first (third) difference threshold 730 and less than or equal to the second (fourth) difference threshold 740, the target weight data can be determined according to the curve between the first (third) difference threshold 730 and the second (fourth) difference threshold 740, and then the global (local) color data obtained by applying the initial global (local) color gain to the original image and the spectral perception data (of each detection area) can be interpolated by the target weight data to determine the target global (local) color gain. When it is determined that the difference data is greater than the second (fourth) difference threshold 740, the weight data can be 0 at this time, and the spectral perception data in the (target detection area) is used as the target global (local) color gain (within the target block area).

[0107] Figure 8 A schematic diagram schematically illustrates a principle of calculating a target local color gain in an exemplary embodiment of the present disclosure.

[0108] refer to Figure 8 As shown, the original image is subjected to a first tone mapping process to obtain an initial local color gain 810 , and the spectral perception data 820 corresponding to the original image can be determined by a spectral perception device, and the target local color gain 830 is obtained based on the initial local color gain 810 and the spectral perception data 820 .

[0109] For example, when it is determined that the difference data between the color data after the initial local color gain 810 in the block area of ​​the 1st row and the 1st column and the spectral perception data of the 1st row and the 1st column is greater than the third difference threshold and less than or equal to the fourth difference threshold, the target weight data can be determined, and then the color data after the initial local color gain 810 in the block area of ​​the 1st row and the 1st column and the spectral perception data of the 1st row and the 1st column can be interpolated by the target weight data to determine the target local color gain 831 of the 1st row and the 1st column.

[0110] When it is determined that the difference data between the color data after the initial local color gain in the block area of ​​the 1st row and the 5th column and the spectral perception data of the 1st row and the 5th column is less than or equal to the third difference threshold, the initial local color gain in the block area of ​​the 1st row and the 5th column can be directly used as the target local color gain 832 of the 1st row and the 5th column.

[0111] When it is determined that the difference data between the color data after the initial local color gain in the block area of ​​the 2nd row and the 5th column and the spectral perception data of the 2nd row and the 5th column is greater than the fourth difference threshold, the spectral perception data of the 2nd row and the 5th column can be used as the target local color gain 833 of the 2nd row and the 5th column.

[0112] It can be understood that the target local color gain in other block areas is calculated in the same manner as the target local color gain 831 in the first row and the first column, the target local color gain 832 in the first row and the fifth column, and the target local color gain 833 in the second row and the fifth column, and will not be described in detail here. Based on this, the target local color gain corresponding to the original image can be obtained. Of course, Figure 8 This is merely an illustrative example and should not constitute any particular limitation to this exemplary embodiment.

[0113] In summary, in this exemplary embodiment, the initial color gain obtained by the original image after the first tone mapping process can be determined, and the spectral perception data corresponding to the original image can be obtained. Then, the target color gain can be determined based on the spectral perception data and the initial color gain. Finally, the original image can be subjected to the second tone mapping process according to the target color gain to obtain the target image. The accurate color data corresponding to the original image is obtained through the spectral perception data, and the target color gain that is finally applied to the original image is determined in combination with the initial color gain. While realizing the tone mapping of the image, the accuracy of the color restoration of the original image can be effectively improved, and the problem of image color loss or distortion caused by tone mapping based on brightness adjustment can be avoided, thereby improving the color restoration effect of the original image and ensuring the image quality of the target image.

[0114] It should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0115] For further reference, Fig. 9 As shown, in the embodiment of this example, an image tone mapping device 900 is also provided, including an initial color gain determination module 910, a spectrum perception data acquisition module 920, a target color gain determination module 930 and an image tone mapping module 940. Among them:

[0116] The initial color gain determination module 910 is used to obtain an original image and determine an initial color gain of the original image obtained by a first tone mapping process;

[0117] The spectral perception data acquisition module 920 is used to acquire the spectral perception data corresponding to the original image;

[0118] The target color gain determination module 930 is used to determine the target color gain based on the spectral perception data and the initial color gain;

[0119] The image tone mapping module 940 is used to perform a second tone mapping process on the original image according to the target color gain to obtain a target image.

[0120] In an exemplary embodiment, the spectrum sensing data acquisition module 920 may be used to:

[0121] Acquiring spectral perception data corresponding to the original image through a preset spectral perception device;

[0122] The spectral sensing device includes a spectral sensor array composed of at least two spectral sensors, and the spectral sensor array can generate spectral sensing data corresponding to at least two detection areas.

[0123] In an exemplary embodiment, the target color gain may include a target global color gain, and the target color gain determination module 930 may be configured to:

[0124] Performing global tone mapping on the original image to determine an initial global color gain;

[0125] Comparing the initial global color gain with the spectral perception data to obtain a comparison result;

[0126] A target global color gain is determined according to the comparison result.

[0127] In an exemplary embodiment, the target color gain determination module 930 may be configured to:

[0128] Obtaining a preset first difference threshold and a preset second difference threshold;

[0129] Applying the initial global color gain to the original image to obtain global color data, and determining difference data between the global color data and the spectral perception data;

[0130] If it is determined that the difference data is less than or equal to the first difference threshold, the initial global color gain is used as the target global color gain.

[0131] In an exemplary embodiment, the target color gain determination module 930 may also be used to:

[0132] If it is determined that the difference data is greater than the first difference threshold and less than or equal to the second difference threshold, obtaining a difference weight curve;

[0133] Determining target weight data based on the difference data and the difference weight curve;

[0134] An interpolation calculation is performed on the global color data and the spectral perception data according to the target weight data to determine a target global color gain.

[0135] In an exemplary embodiment, the target color gain determination module 930 may also be used to:

[0136] If it is determined that the difference data is greater than the second difference threshold, a target global color gain is calculated according to the spectral perception data.

[0137] In an exemplary embodiment, the target color gain may include a target local color gain, and the target color gain determination module 930 may be configured to:

[0138] Determine a block area array corresponding to the original image, wherein the block areas in the block area array are consistent in size and position with the detection areas in the spectrum sensing device;

[0139] Counting pixel information corresponding to the block areas of the original image, and determining an initial local color gain of each of the block areas according to the pixel information;

[0140] Comparing the initial local color gain of each of the block areas with the spectral perception data of the detection area corresponding to the block area to obtain a comparison result;

[0141] A target local color gain is determined according to the comparison result.

[0142] In an exemplary embodiment, the target color gain determination module 930 may be configured to:

[0143] Obtaining a preset third difference threshold and a preset fourth difference threshold;

[0144] Applying the initial local color gain to the original image to obtain local color data of each of the block areas, and determining difference data between the local color data and the spectral perception data;

[0145] If it is determined that the difference data is less than or equal to the third difference threshold, the initial local color gain is used as the target local color gain.

[0146] In an exemplary embodiment, the target color gain determination module 930 may also be used to:

[0147] If it is determined that the difference data is greater than the third difference threshold and less than or equal to the fourth difference threshold, obtaining a difference weight curve;

[0148] Determining target weight data based on the difference data and the difference weight curve;

[0149] An interpolation calculation is performed on the local color data and the spectral perception data according to the target weight data to determine a target local color gain.

[0150] In an exemplary embodiment, the target color gain determination module 930 may also be used to:

[0151] If it is determined that the difference data is greater than the fourth difference threshold, a target local color gain is calculated according to the spectral perception data.

[0152] The specific details of each module in the above device have been described in detail in the implementation method of the method part. The undisclosed details can be found in the implementation method of the method part, so they will not be repeated here.

[0153] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0154] An exemplary embodiment of the present disclosure further provides an electronic device. Generally, the electronic device may include a processor and a memory, the memory is used to store executable instructions of the processor, and the processor is configured to perform the above-mentioned image tone mapping method by executing the executable instructions.

[0155] Below Fig.10 The mobile terminal 1000 in FIG. 1 is taken as an example to exemplify the structure of the electronic device. It should be understood by those skilled in the art that, in addition to the components specifically used for mobile purposes, Fig.10 The construction in can also be applied to fixed type equipment.

[0156] like Fig.10 As shown, the mobile terminal 1000 may specifically include: a processor 1001, a memory 1002, a bus 1003, a mobile communication module 1004, an antenna 1, a wireless communication module 1005, an antenna 2, a display screen 1006, a camera module 1007, an audio module 1008, a power module 1009 and a sensor module 1010.

[0157] Processor 1001 may include one or more processing units, for example: processor 1001 may include AP (Application Processor), modem processor, GPU (Graphics Processing Unit), ISP (Image Signal Processor), controller, encoder, decoder, DSP (Digital Signal Processor), baseband processor and / or NPU (Neural-Network Processing Unit), etc.

[0158] The encoder can encode (i.e. compress) an image or video to reduce the data size for easy storage or transmission. The decoder can decode (i.e. decompress) the encoded data of an image or video to restore the image or video data. The mobile terminal 1000 can support one or more encoders and decoders, such as: image formats such as JPEG (Joint Photographic Experts Group), PNG (Portable Network Graphics), BMP (Bitmap), and video formats such as MPEG (Moving Picture Experts Group) 1, MPEG10, H.1063, H.1064, and HEVC (High Efficiency Video Coding).

[0159] The processor 1001 may be connected to the memory 1002 or other components via a bus 1003 .

[0160] The memory 1002 may be used to store computer executable program codes, which may include instructions. The processor 1001 executes various functional applications and data processing of the mobile terminal 1000 by running the instructions stored in the memory 1002. The memory 1002 may also store application data, such as images, videos, and other files.

[0161] The communication function of the mobile terminal 1000 can be implemented by the mobile communication module 1004, antenna 1, wireless communication module 1005, antenna 2, modulation and demodulation processor and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. The mobile communication module 1004 can provide 3G, 4G, 5G and other mobile communication solutions applied to the mobile terminal 1000. The wireless communication module 1005 can provide wireless communication solutions such as wireless LAN, Bluetooth, near field communication, etc. applied to the mobile terminal 1000.

[0162] The display screen 1006 is used to implement display functions, such as displaying user interfaces, images, videos, etc. The camera module 1007 is used to implement shooting functions, such as shooting images, videos, etc. The audio module 1008 is used to implement audio functions, such as playing audio, collecting voice, etc. The power module 1009 is used to implement power management functions, such as charging the battery, powering the device, monitoring the battery status, etc.

[0163] The sensor module 1010 may include one or more sensors for implementing corresponding sensing detection functions. For example, the sensor module 1010 may include an inertial sensor for detecting the motion posture of the mobile terminal 1000 and outputting inertial sensing data.

[0164] The exemplary embodiments of the present disclosure also provide a computer-readable storage medium on which a program product capable of implementing the above-mentioned method of the present specification is stored. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of the present specification.

[0165] It should be noted that the computer-readable medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0166] In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0167] In addition, program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0168] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0169] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image tone mapping method, characterized in that: include: Acquire an original image, and determine an initial color gain of the original image obtained by a first tone mapping process; Acquire spectral perception data corresponding to the original image; Determining a target color gain based on the spectral perception data and the initial color gain; Performing a second tone mapping process on the original image according to the target color gain to obtain a target image; The initial color gain includes an initial global color gain, the target color gain includes a target global color gain, the first tone mapping includes a global tone mapping, and the determining the target color gain based on the spectral perception data and the initial color gain includes: Comparing the initial global color gain with the spectral perception data to obtain a comparison result, and determining a target global color gain according to the comparison result; wherein the initial global color gain is determined by performing global tone mapping on the original image; The comparing the initial global color gain with the spectral perception data to obtain a comparison result, and determining a target global color gain according to the comparison result, includes: obtaining a preset first difference threshold and a preset second difference threshold; applying the initial global color gain to the original image to obtain global color data, and determining difference data between the global color data and the spectral perception data; if it is determined that the difference data is less than or equal to the first difference threshold, using the initial global color gain as the target global color gain, wherein the first difference threshold is less than the second difference threshold.

2. The method according to claim 1, characterized in that The acquiring spectral perception data corresponding to the original image includes: Acquiring spectral perception data corresponding to the original image through a preset spectral perception device; The spectral sensing device includes a spectral sensor array composed of at least two spectral sensors, and the spectral sensor array can generate spectral sensing data corresponding to at least two detection areas.

3. The method according to claim 1, characterized in that The determining of the target global color gain according to the comparison result further includes: If it is determined that the difference data is greater than the first difference threshold and less than or equal to the second difference threshold, obtaining a difference weight curve; Determining target weight data based on the difference data and the difference weight curve; An interpolation calculation is performed on the global color data and the spectral perception data according to the target weight data to determine a target global color gain.

4. The method according to claim 3, characterized in that The determining of the target global color gain according to the comparison result further includes: If it is determined that the difference data is greater than the second difference threshold, a target global color gain is calculated according to the spectral perception data.

5. The method according to claim 1 or 2, characterized in that: The target color gain further includes a target local color gain, the initial color gain further includes an initial local color gain, the first tone mapping further includes a local tone mapping, and the determining the target color gain based on the spectral perception data and the initial color gain further includes: Comparing the initial local color gain of each block area of ​​the original image with the spectrum perception data of the detection area corresponding to the block area of ​​the spectrum perception device to obtain a comparison result; The target local color gain is determined according to the comparison result between the initial local color gain and the spectral perception data; wherein the initial local color gain is determined by performing local tone mapping on the original image, and the block areas in the block area array of the original image are consistent in size and position with the detection area in the spectral perception device.

6. The method according to claim 5, characterized in that The initial local color gain of each block area of ​​the original image is compared with the spectrum perception data of the detection area corresponding to the block area of ​​the spectrum perception device to obtain a comparison result; Determining a target local color gain according to the comparison result between the initial local color gain and the spectral perception data includes: Obtaining a preset third difference threshold and a preset fourth difference threshold; Applying the initial local color gain to the original image to obtain local color data of each of the block areas, and determining difference data between the local color data and the spectral perception data; If it is determined that the difference data between the local color data and the spectral perception data is less than or equal to the third difference threshold, the initial local color gain is used as the target local color gain, wherein the third difference threshold is less than a fourth difference threshold.

7. The method according to claim 6, characterized in that The determining of the target local color gain according to the comparison result further includes: If it is determined that the difference data between the local color data and the spectral perception data is greater than the third difference threshold and less than or equal to the fourth difference threshold, obtaining a difference weight curve; Determining target weight data based on the difference data between the local color data and the spectral perception data and the difference weight curve; An interpolation calculation is performed on the local color data and the spectral perception data according to the target weight data to determine a target local color gain.

8. The method according to claim 7, characterized in that The determining of the target local color gain according to the comparison result further includes: If it is determined that the difference data between the local color data and the spectral perception data is greater than the fourth difference threshold, a target local color gain is calculated according to the spectral perception data.

9. An image tone mapping device, characterized in that: include: An initial color gain determination module, used to obtain an original image and determine an initial color gain obtained by processing the original image through a first tone mapping process; A spectral perception data acquisition module, used to acquire spectral perception data corresponding to the original image; A target color gain determination module, configured to determine a target color gain based on the spectral perception data and the initial color gain; An image tone mapping module, configured to perform a second tone mapping process on the original image according to the target color gain to obtain a target image; The initial color gain includes an initial global color gain, the target color gain includes a target global color gain, the first tone mapping includes a global tone mapping, and the determining the target color gain based on the spectral perception data and the initial color gain includes: Comparing the initial global color gain with the spectral perception data to obtain a comparison result, and determining a target global color gain according to the comparison result; wherein the initial global color gain is determined by performing global tone mapping on the original image; The comparing the initial global color gain with the spectral perception data to obtain a comparison result, and determining a target global color gain according to the comparison result, includes: obtaining a preset first difference threshold and a preset second difference threshold; applying the initial global color gain to the original image to obtain global color data, and determining difference data between the global color data and the spectral perception data; if it is determined that the difference data is less than or equal to the first difference threshold, using the initial global color gain as the target global color gain, wherein the first difference threshold is less than the second difference threshold.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

11. An electronic device, characterized in that: include: processor; A memory, configured to store executable instructions of the processor; wherein the processor is configured to perform the method of any one of claims 1 to 8 by executing the executable instructions; as well as The spectral sensing device is used to obtain spectral sensing data corresponding to the original image.

Images