Image processing method, device, equipment and storage medium thereof

Abstract

The present invention provides an image processing method, comprising the steps of: providing a plurality of original images; obtaining blurred images based on the original images, obtaining detail information based on the difference between the original images and the blurred images, and obtaining detail enhancement values ​​based on the detail information; performing a first mapping on the original images to obtain a first mapped image, performing a second mapping on the original images to obtain a second mapped image, so as to retain the details of the second mapped image, obtaining fusion weights based on the details and brightness of the first mapped image and the details and brightness of the second mapped image, obtaining weighted fused pixel values ​​based on the fusion weights; and obtaining output image pixel values ​​based on the detail enhancement values ​​and the weighted fused pixel values. The present invention enhances the details of an image and solves the problem of detail loss in highlights of an image after global tone mapping. The present invention also provides an apparatus, device, and storage medium for implementing the image processing method.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image processing method and an apparatus, device and storage medium thereof. Background Art

[0002] Traditional sensors have a dynamic range of 10 or 12 bits of data. With existing technology, the actual contrast can reach as high as 20 bits. This results in loss of detail and overexposure in bright areas when displaying high-dynamic range (HDR) images. Furthermore, because traditional displays use 24-bit true color, with each channel being 8 bits, and high-dynamic range images using 12 bits or higher per color channel, HDR images cannot be displayed on conventional displays. Therefore, tone mapping is required.

[0003] Tone mapping compresses high-dynamic-range images, for example, by reducing the image size of a video stream to a range that can be displayed on traditional displays. An image processed by a tone mapping algorithm should produce a subjective perception consistent with the real scene. In other words, tone mapping not only compresses the dynamic range but also maximizes the preservation of color, contrast, and detail in images with the highest dynamic range.

[0004] Conventional tone mapping is categorized into global tone mapping and local tone mapping. Global tone mapping applies the same transformation to all pixels in an image. This method is fast and simple to implement, but it can lead to loss of detail, especially in scenes with large contrast ranges, resulting in poor rendering of highlights. Local tone mapping adjusts different areas of an image differently, but compared to global tone mapping, it has the disadvantages of increased complexity, haloing, and unnatural images.

[0005] Chinese patent publication number CN 108986174 A discloses a global tone mapping method and system for high dynamic range (HDR) images. The method includes: obtaining images of a property corresponding to each point in the property at different exposure times; filtering out the maximum and minimum brightness values ​​from the brightness values ​​corresponding to all pixels in the property image; normalizing the brightness value of each pixel based on the brightness value, the maximum, and the minimum brightness values ​​to obtain a normalized brightness value for each pixel; and performing tone mapping on the property image based on the normalized brightness value of each pixel. However, this invention does not address the problem of detail loss in highlights of the image after global tone mapping.

[0006] Therefore, it is necessary to provide an image processing method and its apparatus, device and storage medium to solve the above-mentioned problems in the prior art. Summary of the Invention

[0007] The object of the present invention is to provide an image processing method and apparatus, device and storage medium thereof, so as to solve the problem of detail loss in highlight areas of an image after global tone mapping.

[0008] To achieve the above object, the image processing method of the present invention comprises the steps of:

[0009] Provide several original images;

[0010] performing a blurring operation on the original image to obtain a blurred image, obtaining detail information based on a difference between the original image and the blurred image, and obtaining a detail enhancement value based on the detail information, wherein the detail enhancement value is used to enhance details of a fused image;

[0011] Performing a first mapping on the original image to obtain a first mapped image, so that the overall brightness of the first mapped image is within a first brightness range;

[0012] performing a second mapping on the original image to obtain a second mapped image, so that the overall brightness of the second mapped image is within a second brightness range to retain details of the second mapped image, wherein the first brightness range and the second brightness range are two different brightness value intervals;

[0013] Obtaining a fusion weight according to the details and brightness of the first mapped image and the details and brightness of the second mapped image, and obtaining a weighted fused pixel value according to the fusion weight;

[0014] An output image pixel value is obtained according to the detail enhancement value and the weighted fused pixel value.

[0015] The image processing method of the present invention has the following beneficial effects:

[0016] The image processing method of the present invention performs different processing on the original image to obtain the blurred image, the first mapping image and the second mapping image respectively, obtains detail information and detail enhancement values ​​through the blurred image to enhance the details of the image; obtains the second mapping image through processing the original image to retain the details of the second mapping image, obtains fusion weights and weighted fused pixel values ​​through the details and brightness of the first mapping image and the details and brightness of the second mapping image; obtains output image pixel values ​​through the detail enhancement values ​​and the weighted fused pixel values, thereby enhancing the details of the fused image, and the output image pixel values ​​obtained through the detail enhancement values ​​and the weighted fused pixel values ​​make the final image more natural, thereby solving the problem of detail loss in highlights of the image after global tone mapping; the algorithm operation has low complexity and is easy to implement, avoiding the occurrence of halo and unnatural image phenomena.

[0017] Optionally, the step of obtaining detail information based on the difference between the original image and the blurred image includes:

[0018] An image subtraction operation is performed on the original image and the blurred image to obtain the detail information.

[0019] Optionally, the step of obtaining a detail enhancement value according to the detail information includes:

[0020] A first gain multiplier for detail superposition is obtained based on the brightness of the blurred image, and the detail information is multiplied by the first gain multiplier to obtain the detail enhancement value. This advantageously provides the advantage that the detail enhancement value obtained by calculating the obtained detail information and the first gain multiplier facilitates subsequent enhancement of image details.

[0021] Optionally, the step of obtaining a first gain multiple for detail superposition according to the brightness of the blurred image includes:

[0022] Setting a detail intensity transition value, a maximum gain multiple, and a first threshold for detail superposition, and obtaining a second threshold based on the first threshold, wherein the maximum gain multiple is greater than the detail intensity transition value, and the second threshold is greater than the first threshold;

[0023] Obtaining the brightness of the blurred image, and obtaining an intermediate gain multiplier according to the brightness of the blurred image, the first threshold, and the detail intensity transition value, wherein the intermediate gain multiplier is greater than 0 and less than the maximum gain multiplier;

[0024] The brightness of the blurred image is compared with the first and second thresholds to obtain a comparison result, and the first gain multiplier is obtained based on the comparison result. This advantageously provides an algorithm for calculating the first gain multiplier, thereby improving the reliability of the first gain multiplier and enhancing image detail in a subsequent image fusion step, resulting in a more natural fused image.

[0025] Optionally, the step of obtaining the second threshold according to the first threshold includes:

[0026] The first parameter is left-shifted by the detail intensity transition value to obtain an addend, and an addition operation is performed on the first threshold and the addend to obtain the second threshold.

[0027] Optionally, the step of obtaining an intermediate gain multiple according to the brightness of the blurred image, the first threshold, and the detail intensity transition value includes:

[0028] A subtraction operation is performed on the brightness of the blurred image and the first threshold to obtain a first difference, a subtraction operation is performed on the weight bit width of the blurred image brightness and the detail intensity transition value to obtain a second difference, and the first difference is shifted left by the second difference bit to obtain the intermediate gain multiplier.

[0029] Optionally, the step of obtaining the first gain multiple according to the comparison result includes:

[0030] When the brightness of the blurred image is less than the first threshold, the first gain multiplier is equal to 0;

[0031] When the brightness of the blurred image is greater than or equal to the first threshold and less than the second threshold, the first gain multiple is equal to the intermediate gain multiple;

[0032] When the brightness of the blurred image is greater than or equal to the second threshold, the first gain multiple is equal to the maximum gain multiple.

[0033] Optionally, the fusion weight includes a first fusion weight and a second fusion weight, and the step of obtaining the fusion weight according to the details and brightness of the first mapped image and the details and brightness of the second mapped image includes:

[0034] The first fusion weight is obtained according to the details and brightness of the first mapped image, and the second fusion weight is obtained according to the details and brightness of the second mapped image.

[0035] Optionally, the step of obtaining the first fusion weight according to the details and brightness of the first mapped image includes:

[0036] A first brightness weight is obtained according to the brightness of the first mapped image, a first detail weight is obtained according to the detail of the first mapped image, and a weight multiplication operation is performed on the first brightness weight and the first detail weight to obtain the first fusion weight.

[0037] Optionally, the step of obtaining the second fusion weight according to the details and brightness of the second mapped image includes:

[0038] A second brightness weight is obtained according to the brightness of the second mapped image, a second detail weight is obtained according to the detail of the second mapped image, and a weight multiplication operation is performed on the second brightness weight and the second detail weight to obtain the second fusion weight.

[0039] Optionally, the step of obtaining weight-fused pixel values ​​according to the fusion weights includes:

[0040] The final weight of the first image is obtained based on the first fusion weight and the second fusion weight, and the weighted fused pixel value is obtained based on the final weight of the first image, the first mapped image, and the second mapped image. This advantageously provides a more natural image by calculating the weighted fused pixel value after obtaining the first fusion weight and the fusion weight.

[0041] Optionally, the step of obtaining a final weight of the first image according to the first fusion weight and the second fusion weight includes:

[0042] A weight addition operation is performed on the first fusion weight and the second fusion weight to obtain a weight sum, and a division operation is performed on the first fusion weight and the weight sum to obtain a final weight of the first image.

[0043] Optionally, the step of obtaining the weighted fused pixel value according to the final weight of the first image, the first mapped image, and the second mapped image includes:

[0044] performing a multiplication operation on the final weight of the first image and the first mapped image to obtain a first pixel value;

[0045] performing a subtraction operation on a second parameter and a final weight value of the first image to obtain an intermediate value, and performing a multiplication operation on the intermediate value and the second mapped image to obtain a second pixel value;

[0046] An addition operation is performed on the first pixel value and the second pixel value to obtain the weighted fusion pixel value.

[0047] Optionally, the step of obtaining an output image pixel value according to the detail enhancement value and the weighted fused pixel value includes:

[0048] An addition operation is performed on the detail enhancement value and the weighted fused pixel value to obtain the output image pixel value. This advantageously provides that, by performing the addition operation on the detail enhancement value and the weighted fused pixel value to obtain the output image pixel value, the detail enhancement value enhances image details, and the output image pixel value makes the image more natural.

[0049] The present invention also provides an image processing device, comprising:

[0050] An image acquisition module is used to provide a plurality of original images and perform a blurring operation on the original images to obtain blurred images;

[0051] The image acquisition module includes an image mapping unit configured to perform a first mapping on the original image to obtain a first mapped image, such that the overall brightness of the first mapped image is within a first brightness range, and to perform a second mapping on the original image to obtain a second mapped image, such that the overall brightness of the second mapped image is within a second brightness range to preserve details of the second mapped image, wherein the first brightness range and the second brightness range are two different brightness value intervals;

[0052] a detail calculation module, configured to obtain detail information based on the difference between the original image and the blurred image, and to obtain a detail enhancement value based on the detail information, wherein the detail enhancement value is used to enhance the details of the fused image;

[0053] a weight calculation module, configured to obtain a fusion weight according to the details and brightness of the first mapped image and the details and brightness of the second mapped image;

[0054] The pixel value calculation module is used to obtain the weighted fused pixel value according to the fusion weight, and to obtain the output image pixel value according to the detail enhancement value and the weighted fused pixel value.

[0055] The image processing device of the present invention has the following beneficial effects:

[0056] The image acquisition module acquires the original image, the blurred image and the first mapped image, and acquires the second mapped image to retain the details of the second mapped image. The detail calculation module acquires the detail information and the detail enhancement value to enhance the details of the fused image. The weight calculation module calculates the fusion weight. The pixel value calculation module calculates the pixel value after weight fusion and the output image pixel value, thereby making the final image more natural and solving the problem of detail loss in the highlight of the image after global tone mapping.

[0057] The present invention also provides a device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein the image processing method is implemented when the processor executes the program.

[0058] The present invention also provides a storage medium on which a program is stored. When the program is executed by a processor, the image processing method is implemented.

[0059] The device and storage medium of the present invention both have the following beneficial effects:

[0060] Since the device and the storage medium are both used to implement the image processing method, the details of the image are enhanced, making the image more natural. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 is a flow chart of an image processing method according to an embodiment of the present invention;

[0062] Figure 2 is a functional relationship diagram of the first gain multiple and the brightness of the blurred image according to an embodiment of the present invention;

[0063] Figure 3 is a functional relationship diagram of the first brightness weight and the brightness of the first mapped image according to an embodiment of the present invention;

[0064] Figure 4 is a functional relationship diagram of the first detail weight and the detail of the first mapped image according to an embodiment of the present invention;

[0065] Figure 5 is a functional relationship diagram of the second brightness weight and the brightness of the second mapped image according to an embodiment of the present invention;

[0066] Figure 6 is a functional relationship diagram of the second detail weight and the detail of the second mapped image according to an embodiment of the present invention;

[0067] Figure 7 FIG. 4 is a structural block diagram of an image processing device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be the common meanings understood by people with ordinary skills in the field to which the present invention belongs. The words "including" and similar words used in this article mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects.

[0069] In order to solve the problems existing in the prior art, the embodiment of the present invention provides an image processing method. Figure 1 FIG1 is a flow chart of an image processing method according to an embodiment of the present invention. Figure 1 , the image processing method of the present invention comprises the steps of:

[0070] S0: Provide several original images;

[0071] S1: performing a blurring operation on the original image to obtain a blurred image, obtaining detail information based on a difference between the original image and the blurred image, and obtaining a detail enhancement value based on the detail information, wherein the detail enhancement value is used to enhance details of a fused image;

[0072] S2: performing a first mapping on the original image to obtain a first mapped image, so that the overall brightness of the first mapped image is within a first brightness range;

[0073] S3: performing a second mapping on the original image to obtain a second mapped image, so that the overall brightness of the second mapped image is within a second brightness range to retain details of the second mapped image, and the first brightness range and the second brightness range are two different brightness value intervals;

[0074] S4: obtaining a fusion weight according to the details and brightness of the first mapped image and the details and brightness of the second mapped image, and obtaining a weighted fusion pixel value according to the fusion weight;

[0075] S5: Obtain an output image pixel value according to the detail enhancement value and the weighted fused pixel value.

[0076] The advantages of the image processing method of the present invention are: the image processing method of the present invention performs different processing on the original image to obtain the blurred image, the first mapping image and the second mapping image respectively, obtains detail information and detail enhancement values ​​through the blurred image to enhance the details of the fused image; obtains the second mapping image by processing the original image to retain the details of the second mapping image, obtains fusion weights and weighted fused pixel values ​​through the details and brightness of the first mapping image and the details and brightness of the second mapping image; obtains output image pixel values ​​through the detail enhancement values ​​and the weighted fused pixel values, thereby achieving enhancement of the details of the fused image, and the output image pixel values ​​obtained through the detail enhancement values ​​and the weighted fused pixel values ​​make the final image more natural, thereby solving the problem of detail loss in highlights of the image after global tone mapping; the algorithm operation has low complexity and is easy to implement, avoiding the occurrence of halo and unnatural image phenomena.

[0077] In some embodiments, the original image is a high dynamic range image.

[0078] In some optional embodiments, the maximum value of the first brightness range is smaller than the maximum value of the second brightness range.

[0079] In some optional embodiments, in step S2, the specific steps of obtaining the first mapping image are:

[0080] The original image is globally mapped to adjust the overall brightness of the original image to within the first brightness domain to obtain a first mapped image; specifically, the overall brightness of the original image is adjusted to within the first brightness domain through curve stretching mapping to obtain the first mapped image, and the high-bitwidth original image is mapped into a low-bitwidth first mapped image, and the low-bitwidth is consistent with the first brightness domain, so that the pixel points at the brighter positions of the image are adjusted to a larger extent, and the pixel points at the darker positions of the image are adjusted to a smaller extent.

[0081] In some optional embodiments, in step S3, the specific steps of obtaining the second mapping image are:

[0082] The original image is globally mapped to adjust the overall brightness of the original image to within the second brightness domain to obtain a second mapped image. Specifically, the overall brightness of the original image is adjusted to within the second brightness domain through curve stretching mapping to obtain the second mapped image, so as to retain the details of the second mapped image.

[0083] In some embodiments, the brightness value range of the first brightness domain is 16-24 bits, and the brightness value range of the second brightness domain is 10-12 bits. 16 bits means that the brightness range of the image is 0-(2 16 -1), 24bit means the brightness range of the image is 0-(2 24 -1), 10bit means the brightness range of the image is 0-(2 10 -1), 12bit means the brightness range of the image is 0-(2 12 -1).

[0084] In some optional embodiments, the order of the above-mentioned steps S1, S2 and S3 is not strictly limited, that is, steps S1 to S3 of the image processing method of the present invention are not limited to first performing a blurring operation on the original image to obtain a blurred image, detail information and detail enhancement value, and then obtaining a first mapping image based on the original image, and finally obtaining a second mapping image based on the original image.

[0085] In some embodiments, the above steps S1, S2 and S3 can be performed simultaneously to improve the calculation efficiency of the algorithm.

[0086] In other embodiments, steps S1 to S3 may also be to first obtain a first mapped image based on the original image, then obtain a second mapped image based on the original image, and finally perform a blurring operation on the original image to obtain a blurred image, detail information and detail enhancement value.

[0087] As an optional embodiment of the present invention, in step S1, the step of obtaining detail information based on the difference between the original image and the blurred image includes:

[0088] An image subtraction operation is performed on the original image and the blurred image to obtain the detail information.

[0089] In some optional embodiments, a blurring operation is performed on the original image to obtain the blurred image; specifically, a local filtering operation is performed on the original image to obtain the blurred image; and a calculation formula for performing an image subtraction operation on the original image and the blurred image to obtain the detail information is:

[0090] I_detial=I_orig-I_blur

[0091] Among them, I_detial is the detail information, I_orig is the original image, and I_blur is the blurred image.

[0092] As an optional implementation manner of the present invention, in step S1, the step of obtaining a detail enhancement value according to the detail information includes:

[0093] A first gain multiplier for detail superposition is obtained based on the brightness of the blurred image, and the detail information is multiplied by the first gain multiplier to obtain the detail enhancement value. Advantageously, obtaining the detail enhancement value by calculating the obtained detail information and the first gain multiplier facilitates subsequent enhancement of image details.

[0094] As an optional embodiment of the present invention, the step of obtaining a first gain multiple for detail superposition according to the brightness of the blurred image includes:

[0095] Setting a detail intensity transition value, a maximum gain multiple, and a first threshold for detail superposition, and obtaining a second threshold based on the first threshold, wherein the maximum gain multiple is greater than the detail intensity transition value, and the second threshold is greater than the first threshold;

[0096] Obtaining the brightness of the blurred image, and obtaining an intermediate gain multiplier according to the brightness of the blurred image, the first threshold, and the detail intensity transition value, wherein the intermediate gain multiplier is greater than 0 and less than the maximum gain multiplier;

[0097] The brightness of the blurred image is compared with the first and second thresholds to obtain a comparison result, and the first gain multiplier is obtained based on the comparison result. This advantageously provides an algorithm for calculating the first gain multiplier, thereby improving the reliability of the first gain multiplier and enhancing image detail in a subsequent image fusion step, resulting in a more natural fused image.

[0098] As an optional implementation manner of the present invention, the step of obtaining the second threshold value according to the first threshold value includes:

[0099] The first parameter is left-shifted by the detail intensity transition value to obtain an addend, and an addition operation is performed on the first threshold and the addend to obtain the second threshold.

[0100] In some embodiments, the first parameter is 1.

[0101] As an optional embodiment of the present invention, the step of obtaining an intermediate gain multiple according to the brightness of the blurred image, the first threshold, and the detail intensity transition value includes:

[0102] A subtraction operation is performed on the brightness of the blurred image and the first threshold to obtain a first difference, a subtraction operation is performed on the weight bit width of the blurred image brightness and the detail intensity transition value to obtain a second difference, and the first difference is shifted left by the second difference bit to obtain the intermediate gain multiplier.

[0103] As an optional implementation manner of the present invention, the step of obtaining the first gain multiple according to the comparison result includes:

[0104] When the brightness of the blurred image is less than the first threshold, the first gain multiplier is equal to 0;

[0105] When the brightness of the blurred image is greater than or equal to the first threshold and less than the second threshold, the first gain multiple is equal to the intermediate gain multiple;

[0106] When the brightness of the blurred image is greater than or equal to the second threshold, the first gain multiple is equal to the maximum gain multiple.

[0107] Figure 2 FIG. 4 is a functional relationship diagram of the first gain factor and the brightness of the blurred image according to an embodiment of the present invention.

[0108] In some optional embodiments, the specific steps of obtaining the first gain multiple according to the brightness of the blurred image are as follows:

[0109] Reference Figure 2 , the functional relationship between the first gain multiplier Gain and the brightness Iblur of the blurred image is expressed in the rectangular coordinate system. The calculation formula of the first gain multiplier is:

[0110]

[0111] Wherein, Gain is the first gain multiplier, Iblur is the brightness of the blurred image; thres0 is the first threshold, x0 is the first detail threshold, GainBit is the detail intensity transition value, WeightBit0 is the weight bit width of the blurred image brightness, GainMax is the maximum gain multiplier, and thres1 is the second threshold; (Iblur-x0)<<(WeightBit0-GainBit) is the intermediate gain multiplier, (Iblur-x0) is the first difference, (WeightBit0-GainBit) is the second difference, (Iblur-x0)<<(WeightBit0-GainBit) means that (Iblur-x0) is shifted left by (WeightBit0-GainBit) bits, (Iblur-x0)<<(WeightBit0-GainBit)=(Iblur-x0)×2 (WeightBit0-GainBit) ;

[0112] Thres1 is calculated based on thres0 and GainBit. The specific relationship formula is:

[0113] thres1=thres0+(1<<GainBit)

[0114] Among them, 1 is the first parameter, 1<<GainBit means shifting 1 left by GainBit, 1<<GainBit=1×2 GainBit .

[0115] In some specific embodiments, the weight bit width WeightBit0 of the blurred image brightness is 10 bits, and the value range of the weight is 0-1024.

[0116] In some other specific embodiments, the weight bit width WeightBit0 of the blurred image brightness is 12 bits, and the value range of the weight is 0-4096.

[0117] In some optional embodiments, a calculation formula for performing a multiplication operation on the detail information and the first gain multiple to obtain the detail enhancement value is:

[0118] Detail = Gain × I_detail

[0119] Among them, Detail is the detail enhancement value, Gain is the first gain multiplier, and I_detial is the detail information.

[0120] As an optional embodiment of the present invention, in step S4, the step of obtaining a fusion weight according to the details and brightness of the first mapped image and the details and brightness of the second mapped image includes:

[0121] The fusion weight includes a first fusion weight and a second fusion weight. The first fusion weight is obtained according to the details and brightness of the first mapped image, and the second fusion weight is obtained according to the details and brightness of the second mapped image.

[0122] As an optional implementation manner of the present invention, the step of obtaining the first fusion weight according to the details and brightness of the first mapped image includes:

[0123] A first brightness weight is obtained according to the brightness of the first mapped image, a first detail weight is obtained according to the detail of the first mapped image, and a weight multiplication operation is performed on the first brightness weight and the first detail weight to obtain the first fusion weight.

[0124] Figure 3 4 is a functional relationship diagram of the first brightness weight and the brightness of the first mapped image according to an embodiment of the present invention.

[0125] In some specific embodiments, referring to Figure 3 , the functional relationship between the first brightness weight W_lum1 and the brightness lum1 of the first mapping image is expressed in the rectangular coordinate system. The calculation formula for obtaining the first brightness weight according to the brightness of the first mapping image is:

[0126]

[0127] Among them, W_lum1 is the first brightness weight, lum1 is the brightness of the first mapped image, LowBit is the low brightness transition value, and HighBit is the high brightness transition value; WeightBit1 is the first weight bit width, specifically the weight bit width of the brightness of the first mapped image; WeightMax1 is the first weight maximum value, specifically the weight maximum value of the brightness of the first mapped image; (lum1-x1)<<(WeightBit1-LowBit) means shifting (lum1-x1) left by (WeightBit1-LowBit) bits; (lum1-x3)<<(WeightBit1-HighBit) means shifting (lum1-x3) left by (WeightBit1-HighBit) bits; x1 is the first brightness threshold, x2 is the second brightness threshold, x3 is the third brightness threshold, and x4 is the fourth brightness threshold. The relationship formula between x2, x1 and LowBit, and the relationship formula between x3, x4 and HighBit are as follows:

[0128] x2=x1+(1<<LowBit);

[0129] x3=x4-(1<<HighBit)

[0130] Among them, 1<<LowBit means shifting 1 left by LowBit, and 1<<HighBit means shifting 1 left by HighBit;

[0131] WeightMax1 is equal to 2 to the power of WeightBit1, that is, the relationship between WeightMax1 and WeightBit1 is as follows:

[0132] WeightMax1=2 WeightBit1 .

[0133] In some specific embodiments, WeightBit1=10, WeightMax1=1024.

[0134] In other specific embodiments, WeightBit1=12, WeightMax1=4096.

[0135] Figure 4FIG. 4 is a functional relationship diagram of the first detail weight and the detail of the first mapped image according to an embodiment of the present invention.

[0136] In some embodiments, the first detail weight is obtained according to the details of the first mapped image. Specifically, the first detail weight is calculated by a gradient operator within the detail range of the current first mapped image. Figure 4 , the functional relationship between the first detail weight W_detail1 and the detail detail1 of the first mapped image is expressed in the rectangular coordinate system. The calculation formula of the first detail weight is as follows:

[0137]

[0138] Among them, W_detail1 is the first detail weight, detail1 is the detail of the first mapped image, WeightBit2 is the second weight bit width, specifically the weight bit width of the detail of the first mapped image; WeightMax2 is the second weight maximum value, the weight maximum value of the detail of the first mapped image; DetailBit is the transition value of the detail intensity of the first mapped image, and its interval length is 2 to the power of detialBit; (detail1-x5)<<(WeightBit2-DetailBit) means shifting (detail1-x5) left by (WeightBit2-DetailBit) bits, x5 is the set second detail threshold, x6 is the set third detail threshold, and the relationship formula between x5 and x6 is as follows:

[0139] x6=x5+(1<<DetailBit)

[0140] Among them, 1<<DetailBit means shifting 1 to the left by DetailBit;

[0141] WeightMax2 is equal to 2 to the power of WeightBit2, that is, the relationship between WeightMax2 and WeightBit2 is as follows:

[0142] WeightMax2=2 WeightBit2 .

[0143] In some specific embodiments, WeightBit2=10, WeightMax2=1024.

[0144] In other specific embodiments, WeightBit2=12, WeightMax2=4096.

[0145] In some embodiments, the calculation process of the detail detail1 of the first mapped image is similar to the above-mentioned detail enhancement value Detail, and the specific steps are:

[0146] A second gain multiplier Gain1 is obtained based on the brightness of the first mapped image. The first mapped image I1 is subtracted from the original image I_orig to obtain first detail information I_detail1. The second gain multiplier Gain1 is multiplied by the first detail information I_detail1 to obtain detail detail1 of the first mapped image. The specific calculation formula is as follows:

[0147] I_detial1=I_orig-I1;

[0148] detail1=Gain1×I_detail1;

[0149] It can be noted that the calculation method of the second gain multiple Gain1 is similar to the calculation method of the first gain multiple Gain, and will not be repeated here.

[0150] In some embodiments, a calculation formula for performing a weight multiplication operation on the first brightness weight and the first detail weight to obtain the first fusion weight is:

[0151] W1=W_lum1×W_detail1

[0152] Among them, W1 is the first fusion weight, W_lum1 is the first brightness weight, and W_detail1 is the first detail weight.

[0153] As an optional implementation manner of the present invention, the step of obtaining the second fusion weight according to the details and brightness of the second mapped image includes:

[0154] A second brightness weight is obtained according to the brightness of the second mapped image, a second detail weight is obtained according to the detail of the second mapped image, and a weight multiplication operation is performed on the second brightness weight and the second detail weight to obtain the second fusion weight.

[0155] Figure 5 FIG. 4 is a functional relationship diagram of the second brightness weight and the brightness of the second mapped image according to an embodiment of the present invention.

[0156] In some specific embodiments, referring to Figure 5 , the functional relationship between the second luminance weight W_lum2 and the luminance lum2 of the second mapped image is expressed in a rectangular coordinate system. The calculation formula for obtaining the second luminance weight according to the luminance of the second mapped image is:

[0157]

[0158] Among them, W_lum2 is the second brightness weight, lum2 is the brightness of the second mapped image, LowBit is the low brightness transition value, and HighBit is the high brightness transition value; WeightBit3 is the third weight bit width, specifically the weight bit width of the brightness of the second mapped image; WeightMax3 is the third weight maximum value, specifically the weight maximum value of the brightness of the second mapped image; (lum2-x1)<<(WeightBit3-LowBit) means shifting (lum2-x1) left by (WeightBit3-LowBit) bits, and (lum2-x3)<<(WeightBit3-HighBit) means shifting (lum2-x3) left by (WeightBit3-HighBit) bits; x1 is the first brightness threshold, x2 is the second brightness threshold, x3 is the third brightness threshold, and x4 is the fourth brightness threshold;

[0159] WeightMax3 is equal to 2 to the power of WeightBit3, that is, the relationship between WeightMax3 and WeightBit3 is as follows:

[0160] WeightMax3=2 WeightBit3 .

[0161] In some specific embodiments, WeightBit3=10, WeightMax3=1024.

[0162] In some other specific embodiments, WeightBit3=12, WeightMax3=4096.

[0163] Figure 6 FIG. 4 is a functional relationship diagram of the second detail weight and the detail of the second mapped image according to an embodiment of the present invention.

[0164] In some specific embodiments, the second detail weight is obtained according to the details of the second mapped image. Specifically, the second detail weight is obtained by calculating the gradient operator within the detail range of the second mapped image. Figure 6 , the functional relationship between the second detail weight W_detail2 and the detail detail2 of the second mapped image is expressed in the rectangular coordinate system. The calculation formula of the second detail weight is:

[0165]

[0166] Among them, W_detail2 is the second detail weight, detail2 is the detail of the second mapped image, WeightBit4 is the fourth weight bit width, specifically the weight bit width of the detail of the second mapped image, DetailBit is the transition value of the detail intensity of the second mapped image; WeightMax4 is the fourth weight maximum value, specifically the weight maximum value of the detail of the second mapped image; (detail2-x5)<<(WeightBit4-DetailBit) means shifting (detail2-x5) left by (WeightBit4-DetailBit) bits; x5 is the set second detail threshold, and x6 is the set third detail threshold.

[0167] WeightMax4 is equal to 2 to the power of WeightBit4, that is, the relationship between WeightMax4 and WeightBit4 is as follows:

[0168] WeightMax4=2 WeightBit4 .

[0169] In some specific embodiments, WeightBit4=10, WeightMax4=1024.

[0170] In other specific embodiments, WeightBit4=12, WeightMax4=4096.

[0171] In some embodiments, the calculation process of the detail detail2 of the second mapped image is similar to the above-mentioned detail enhancement value Detail, and the specific steps are:

[0172] A third gain multiplier Gain2 is obtained based on the brightness of the second mapped image. The second mapped image I2 is subtracted from the original image I_orig to obtain second detail information I_detail2. The third gain multiplier Gain2 is multiplied by the second detail information I_detail2 to obtain detail detail2 of the second mapped image. The specific calculation formula is as follows:

[0173] I_detial2=I_orig-I2;

[0174] detail2 = Gain2 × I_detail2;

[0175] It can be noted that the calculation method of the third gain multiple Gain2 is similar to the calculation method of the first gain multiple Gain, and will not be repeated here.

[0176] In some embodiments, a calculation formula for performing a weight multiplication operation on the second brightness weight and the second detail weight to obtain the second fusion weight is:

[0177] W2=W_lum2×W_detail2

[0178] Among them, W2 is the second fusion weight, W_lum2 is the second brightness weight, and W_detail2 is the second detail weight.

[0179] As an optional embodiment of the present invention, in step S4, the step of obtaining the weighted fused pixel value according to the fusion weight includes:

[0180] The final weight of the first image is obtained based on the first fusion weight and the second fusion weight, and the weighted fused pixel value is obtained based on the final weight of the first image, the first mapped image, and the second mapped image. This method has the advantage that after obtaining the first fusion weight and the fusion weight, the weighted fused pixel value is obtained by calculating the first fusion weight and the fusion weight, and the weighted fused pixel value makes the image more natural.

[0181] As an optional embodiment of the present invention, the step of obtaining the final weight of the first image according to the first fusion weight and the second fusion weight includes:

[0182] A weight addition operation is performed on the first fusion weight and the second fusion weight to obtain a weight sum, and a division operation is performed on the first fusion weight and the weight sum to obtain a final weight of the first image.

[0183] In some specific embodiments, a weighted addition operation is performed on the first fusion weight and the second fusion weight to obtain a weighted sum, and a division operation is performed on the first fusion weight and the weighted sum to obtain a final weight of the first image. The above process is a normalization process of the weights. The calculation formula of the final weight of the first image is as follows:

[0184] W_P=W1 / (W1+W2)

[0185] Among them, W_P is the final weight of the first image, which is specifically the fusion weight of the current pixel point P on the first mapped image; (W1+W2) is the weight sum, W1 is the first fusion weight, and W2 is the second fusion weight.

[0186] As an optional embodiment of the present invention, the step of obtaining the weighted fused pixel value according to the final weight of the first image, the first mapped image, and the second mapped image includes:

[0187] performing a multiplication operation on the final weight of the first image and the first mapped image to obtain a first pixel value;

[0188] performing a subtraction operation on a second parameter and a final weight value of the first image to obtain an intermediate value, and performing a multiplication operation on the intermediate value and the second mapped image to obtain a second pixel value;

[0189] An addition operation is performed on the first pixel value and the second pixel value to obtain the weighted fusion pixel value.

[0190] In some embodiments, the second parameter is 1.

[0191] In some specific embodiments, a calculation formula for obtaining the weighted fused pixel value based on the final weight of the first image, the first mapped image, and the second mapped image is as follows:

[0192] P_Value=W_P×I1+(1-W_P)×I2

[0193] Among them, P_Value is the pixel value after weight fusion, W_P is the final weight of the first image, 1 is the second parameter, (1-W_P) is the intermediate value, I1 is the first mapped image, I2 is the second mapped image, W_P×I1 is the first pixel value, and (1-W_P)×I2 is the second pixel value.

[0194] It can be explained that the above-mentioned acquisition of the weighted fusion pixel value is also the process of image fusion, so the output weighted fusion pixel value is the pixel value of the pixel point of the image after the fusion of pixels.

[0195] As an optional embodiment of the present invention, in step S5, the step of obtaining the output image pixel value according to the detail enhancement value and the weighted fused pixel value includes:

[0196] An addition operation is performed on the detail enhancement value and the weighted fused pixel value to obtain the output image pixel value. Advantageously, by performing the addition operation on the detail enhancement value and the weighted fused pixel value to obtain the output image pixel value, the detail enhancement value enhances image details, and the output image pixel value makes the image more natural.

[0197] In some specific embodiments, the calculation formula for performing an addition operation on the detail enhancement value and the weighted fused pixel value to obtain the output image pixel value is as follows:

[0198] P_out=P_Value+Detail

[0199] Among them, P_out is the output image pixel value, P_Value is the pixel value after weight fusion, and Detail is the detail enhancement value.

[0200] It can be explained that the output image pixel value is the pixel value of the pixel point of the fused image finally output after detail enhancement.

[0201] In some embodiments, the final output fused image is a high dynamic range image.

[0202] Figure 7 FIG. 4 is a structural block diagram of an image processing device according to an embodiment of the present invention.

[0203] The present invention also provides an image processing device, referring to Figure 7 , the image processing device of the present invention includes:

[0204] The image acquisition module 1 is used to provide a plurality of original images and perform a blurring operation on the original images to obtain blurred images;

[0205] The image acquisition module 1 includes an image mapping unit 10, which is configured to perform a first mapping on the original image to obtain a first mapped image, such that the overall brightness of the first mapped image is within a first brightness range, and perform a second mapping on the original image to obtain a second mapped image, such that the overall brightness of the second mapped image is within a second brightness range to preserve details of the second mapped image, wherein the first brightness range and the second brightness range are two different brightness value intervals;

[0206] a detail calculation module 2, configured to obtain detail information based on the difference between the original image and the blurred image, and to obtain a detail enhancement value based on the detail information, wherein the detail enhancement value is used to enhance the details of the image;

[0207] a weight calculation module 3, configured to obtain detail information based on the difference between the original image and the blurred image, and to obtain a detail enhancement value based on the detail information, wherein the detail enhancement value is used to enhance the details of the fused image;

[0208] The pixel value calculation module 4 is configured to obtain a fusion weight according to the details and brightness of the first mapped image and the details and brightness of the second mapped image.

[0209] The advantages of the image processing device of the present invention are: the original image, the blurred image, the first mapped image and the second mapped image are acquired through the image acquisition module 1, the detail information and the detail enhancement value are acquired through the detail calculation module 2 to enhance the details of the image, the fusion weight is calculated through the weight calculation module 3, and the pixel value calculation module 4 calculates and obtains the pixel value after weight fusion and the output image pixel value, thereby making the final image more natural, thereby solving the problem of detail loss in the highlight of the image after global tone mapping.

[0210] In some specific embodiments, referring to Figure 7 , the steps of implementing image processing by the image processing device are:

[0211] (1) A plurality of original images are provided by the image acquisition module 1, and a blurring operation is performed on at least one original image to obtain a blurred image; the original images are globally mapped by the image mapping unit 10, so that the overall brightness of the original images is adjusted to the first brightness domain to obtain a first mapped image; the original images are globally mapped by the image mapping unit 10, so that the overall brightness of the original images is adjusted to the second brightness domain to obtain a second mapped image, so as to retain the details of the second mapped image, and the first brightness domain and the second brightness domain are two different brightness value intervals.

[0212] (2) performing an image subtraction operation on the original image and the blurred image by the detail calculation module 2 to obtain the detail information;

[0213] Obtaining a first gain multiplier for detail superposition according to the details of the blurred image, and performing a multiplication operation on the detail information and the first gain multiplier to obtain the detail enhancement value;

[0214] The specific steps of obtaining the first gain multiple according to the details of the blurred image include:

[0215] A detail intensity transition value, a maximum gain multiple, and a first threshold value for detail superposition are set, and a second threshold value is obtained based on the first threshold value, wherein the maximum gain multiple is greater than the detail intensity transition value, and the second threshold value is greater than the first threshold value. The specific steps for obtaining the second threshold value are:

[0216] shifting the first parameter left by the detail intensity transition value to obtain an addend, and performing an addition operation on the first threshold and the addend to obtain the second threshold;

[0217] Obtaining the brightness of the blurred image, and obtaining an intermediate gain multiplier based on the brightness of the blurred image, the first threshold, and the detail intensity transition value, wherein the intermediate gain multiplier is greater than 0 and less than the maximum gain multiplier. The specific steps of obtaining the intermediate gain multiplier are:

[0218] performing a subtraction operation on the brightness of the blurred image and the first threshold to obtain a first difference, performing a subtraction operation on the weighted bit width of the blurred image brightness and the detail intensity transition value to obtain a second difference, and shifting the first difference left by the second difference bit to obtain the intermediate gain multiplier;

[0219] The brightness of the blurred image is compared with the first threshold and the second threshold to obtain a comparison result, and the first gain multiple is obtained according to the comparison result. The specific steps are:

[0220] When the brightness of the blurred image is less than the first threshold, the first gain multiplier is equal to 0;

[0221] When the brightness of the blurred image is greater than or equal to the first threshold and less than the second threshold, the first gain multiple is equal to the intermediate gain multiple;

[0222] When the brightness of the blurred image is greater than or equal to the second threshold, the first gain multiple is equal to the maximum gain multiple.

[0223] (3) obtaining a fusion weight through the weight calculation module 3, specifically the following steps: obtaining a first brightness weight based on the brightness of the first mapped image, obtaining a first detail weight based on the detail of the first mapped image, and performing a weight multiplication operation on the first brightness weight and the first detail weight to obtain the first fusion weight;

[0224] Obtaining a second brightness weight according to the brightness of the second mapped image, obtaining a second detail weight according to the detail of the second mapped image, and performing a weighted multiplication operation on the second brightness weight and the second detail weight to obtain a second fusion weight;

[0225] A weight addition operation is performed on the first fusion weight and the second fusion weight to obtain a weight sum, and a division operation is performed on the first fusion weight and the weight sum to obtain a final weight of the first image.

[0226] (4) performing a multiplication operation on the final weight of the first image and the first mapped image by the pixel value calculation module 4 to obtain a first pixel value, performing a subtraction operation on 1 and the final weight value of the first image to obtain an intermediate value, performing a multiplication operation on the intermediate value and the second mapped image to obtain a second pixel value, and performing an addition operation on the first pixel value and the second pixel value to obtain the weighted fusion pixel value;

[0227] An addition operation is performed on the detail enhancement value and the weighted fused pixel value to obtain the output image pixel value.

[0228] The present invention also provides a device comprising a memory, a processor and a program stored in the memory and executable on the processor, wherein the image processing method is implemented when the processor executes the program.

[0229] The present invention also provides a storage medium on which a program is stored. When the program is executed by a processor, the image processing method is implemented.

[0230] The advantage of the device and the storage medium of the present invention is that, since both the device and the storage medium are used to implement the image processing method, the details of the image are enhanced, making the image more natural.

[0231] While the embodiments of the present invention have been described in detail above, it will be apparent to those skilled in the art that various modifications and variations of these embodiments are possible. However, it should be understood that such modifications and variations are within the scope and spirit of the present invention as set forth in the claims. Furthermore, the invention described herein is susceptible to other embodiments and may be practiced or implemented in a variety of ways.

Claims

1. An image processing method, characterized in that: Including steps: Providing a plurality of original images, wherein the original images are high dynamic range images; performing a blurring operation on the original image to obtain a blurred image, obtaining detail information based on a difference between the original image and the blurred image, and obtaining a detail enhancement value based on the detail information, wherein the detail enhancement value is used to enhance details of a fused image; Performing a first mapping on the original image to obtain a first mapped image, so that the overall brightness of the first mapped image is within a first brightness range; performing a second mapping on the original image to obtain a second mapped image, so that the overall brightness of the second mapped image is within a second brightness range to retain details of the second mapped image, wherein the first brightness range and the second brightness range are two different brightness value intervals; Obtaining a first fusion weight based on the details and brightness of the first mapped image, obtaining a second fusion weight based on the details and brightness of the second mapped image, and obtaining a weighted fusion pixel value based on the first fusion weight and the second fusion weight, including: obtaining a first brightness weight based on the brightness of the first mapped image, obtaining a first detail weight based on the details of the first mapped image, performing a weighted multiplication operation on the first brightness weight and the first detail weight to obtain the first fusion weight; obtaining a second brightness weight based on the brightness of the second mapped image, obtaining a second detail weight based on the details of the second mapped image, and performing a weighted multiplication operation on the second brightness weight and the second detail weight to obtain the second fusion weight; performing a weighted addition operation on the first fusion weight and the second fusion weight to obtain a weighted sum, and performing a division operation on the first fusion weight and the weighted sum to obtain a final weight of the first image; performing a multiplication operation on the final weight of the first image and the first mapped image to obtain a first pixel value; performing a subtraction operation on a second parameter and the final weight value of the first image to obtain an intermediate value, and performing a multiplication operation on the intermediate value and the second mapped image to obtain a second pixel value, wherein the second parameter is 1; and performing an addition operation on the first pixel value and the second pixel value to obtain the weighted fusion pixel value; An output image pixel value is obtained according to the detail enhancement value and the weighted fused pixel value.

2. The image processing method according to claim 1, wherein: The step of obtaining detail information based on the difference between the original image and the blurred image includes: An image subtraction operation is performed on the original image and the blurred image to obtain the detail information.

3. The image processing method according to claim 1, wherein: The step of obtaining a detail enhancement value according to the detail information includes: A first gain multiple for detail superposition is obtained according to the brightness of the blurred image, and a multiplication operation is performed on the detail information and the first gain multiple to obtain the detail enhancement value.

4. The image processing method according to claim 3, wherein: The step of obtaining a first gain multiple for detail superposition according to the brightness of the blurred image includes: Setting a detail intensity transition value, a maximum gain multiple, and a first threshold for detail superposition, and obtaining a second threshold based on the first threshold, wherein the maximum gain multiple is greater than the detail intensity transition value, and the second threshold is greater than the first threshold; Obtaining the brightness of the blurred image, and obtaining an intermediate gain multiplier according to the brightness of the blurred image, the first threshold, and the detail intensity transition value, wherein the intermediate gain multiplier is greater than 0 and less than the maximum gain multiplier; The brightness of the blurred image is compared with the first threshold and the second threshold to obtain a comparison result, and the first gain multiple is obtained according to the comparison result.

5. The image processing method according to claim 4, wherein: The step of obtaining the second threshold value according to the first threshold value includes: The first parameter is left-shifted by the detail intensity transition value to obtain an addend, and an addition operation is performed on the first threshold and the addend to obtain the second threshold.

6. The image processing method according to claim 5, wherein: The step of obtaining an intermediate gain multiple according to the brightness of the blurred image, the first threshold, and the detail intensity transition value comprises: A subtraction operation is performed on the brightness of the blurred image and the first threshold to obtain a first difference, a subtraction operation is performed on the weight bit width of the blurred image brightness and the detail intensity transition value to obtain a second difference, and the first difference is shifted left by the second difference bit to obtain the intermediate gain multiplier.

7. The image processing method according to claim 6, wherein: The step of obtaining the first gain multiple according to the comparison result includes: When the brightness of the blurred image is less than the first threshold, the first gain multiplier is equal to 0; When the brightness of the blurred image is greater than or equal to the first threshold and less than the second threshold, the first gain multiple is equal to the intermediate gain multiple; When the brightness of the blurred image is greater than or equal to the second threshold, the first gain multiple is equal to the maximum gain multiple.

8. The image processing method according to claim 1, wherein: The step of obtaining an output image pixel value according to the detail enhancement value and the weighted fused pixel value comprises: An addition operation is performed on the detail enhancement value and the weighted fused pixel value to obtain the output image pixel value.

9. An image processing device, characterized in that: include: An image acquisition module is used to provide a plurality of original images and perform a blurring operation on the original images to obtain blurred images; The image acquisition module includes an image mapping unit configured to perform a first mapping on the original image to obtain a first mapped image, such that the overall brightness of the first mapped image is within a first brightness range, and to perform a second mapping on the original image to obtain a second mapped image, such that the overall brightness of the second mapped image is within a second brightness range to preserve details of the second mapped image, wherein the first brightness range and the second brightness range are two different brightness value intervals; a detail calculation module, configured to obtain detail information based on the difference between the original image and the blurred image, and to obtain a detail enhancement value based on the detail information, wherein the detail enhancement value is used to enhance the details of the fused image; a weight calculation module, configured to obtain a first fusion weight based on the details and brightness of the first mapped image, and obtain a second fusion weight based on the details and brightness of the second mapped image, comprising: obtaining a first brightness weight based on the brightness of the first mapped image, obtaining a first detail weight based on the details of the first mapped image, and performing a weighted multiplication operation on the first brightness weight and the first detail weight to obtain the first fusion weight; obtaining a second brightness weight based on the brightness of the second mapped image, obtaining a second detail weight based on the details of the second mapped image, and performing a weighted multiplication operation on the second brightness weight and the second detail weight to obtain the second fusion weight; A pixel value calculation module is used to obtain a weighted fused pixel value based on the first fusion weight and the second fusion weight, and to obtain an output image pixel value based on the detail enhancement value and the weighted fused pixel value, including: performing a weighted addition operation on the first fusion weight and the second fusion weight to obtain a weight sum, performing a division operation on the first fusion weight and the weight sum to obtain a final weight of the first image; performing a multiplication operation on the final weight of the first image and the first mapped image to obtain a first pixel value; performing a subtraction operation on a second parameter and the final weight value of the first image to obtain an intermediate value, performing a multiplication operation on the intermediate value and the second mapped image to obtain a second pixel value, wherein the second parameter is 1; performing an addition operation on the first pixel value and the second pixel value to obtain the weighted fused pixel value.

10. A device comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein: When the processor executes the program, the image processing method according to any one of claims 1 to 8 is implemented.

11. A storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the image processing method according to any one of claims 1 to 8 is implemented.

Images