An image enhancement processing method and system

Abstract

The application provides an image enhancement processing method and system, which comprehensively combines weight calculation based on an image block and weight calculation based on a pixel, takes into account the contribution of overall feature distribution of a multi-exposure image and the contribution of local features or even a single pixel to the image, and provides a train of thought for solving the technical problems of overall feature distribution or image block effects of the existing multi-exposure image. The application has the characteristics of rich fused image information, clear details, simple calculation and the like, and can obtain a high dynamic range image with uniform brightness and good exposure.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an image enhancement processing method and system. BACKGROUND

[0002] The dynamic range of nature is very wide, and the interval between night and day is [10-3cd / m2, 106cd / m2]. For such a wide dynamic range of nature, the human eye has the ability of self-adaptive illumination; in contrast, the dynamic range that can be captured by a general digital camera is only 0-255. The information contained in the low dynamic range image obtained by such capture cannot better represent the high dynamic range scene, and the situation of dimness or brightness may occur. In order to solve this problem, many methods have been proposed in this regard by workers. In order to improve the capture of details of real scenes, different exposure parameters can be used for multiple exposures of the same scene to capture detail information in various brightness ranges, and then the images of different exposure parameters that retain details in different brightness regions are weighted and fused, which is the multi-exposure fusion technology.

[0003] At present, the direct multi-exposure fusion algorithm mainly includes two categories: a fusion method based on irradiance domain and a fusion method based on image domain. The fusion method based on irradiance domain refers to restoring an irradiance image close to the irradiance of the real scene through an image sequence, and mapping to a low dynamic range image with details; the fusion method based on image domain refers to directly extracting effective information of each part for fusion through the image domain. The fusion method based on image domain is divided into a method based on image blocks and a method based on pixels. The difference is whether the effective information is extracted based on image blocks or pixels.

[0004] However, the method based on image blocks is prone to block effects, resulting in uneven brightness; and the method based on pixels is more concerned about the local and cannot extract information from the whole. SUMMARY

[0005] The embodiment of the present application provides an image enhancement processing method and system, and proposes a multi-exposure fusion method combining image block processing and pixel-level processing, so as to solve the problems of detail loss and uneven brightness of the result image when the prior art is used for enhancing and processing natural images.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides an image enhancement processing method, comprising:

[0008] segmenting and performing HSV format processing on the to-be-processed image to obtain an image segmentation image, a saturation distribution image and a brightness distribution image of the to-be-processed image;

[0009] According to the image segmentation map, the saturation distribution map and the brightness distribution map, an image block brightness weight map, an image block contrast weight map and an exposure weight map of the image to be processed are obtained;

[0010] The image block brightness weight map, the image block contrast weight map, the exposure weight map and the image to be processed are subjected to guided filtering and weighted fusion operation, to obtain a sequence base layer image and a sequence detail layer image;

[0011] The sequence base layer image and the sequence detail layer image are merged to obtain a fused image of the image to be processed, so that the image to be processed is enhanced.

[0012] Preferably, the segmentation and HSV format processing of the image to be processed to obtain the image segmentation map, the saturation distribution map and the brightness distribution map of the image to be processed comprise:

[0013] The image to be processed is subjected to superpixel segmentation to obtain the image segmentation map of the image to be processed;

[0014] The image to be processed is subjected to HSV format processing to obtain the saturation distribution map and the brightness distribution map of the image to be processed.

[0015] Preferably, the obtaining of the image block brightness weight map, the image block contrast weight map and the exposure weight map of the image to be processed according to the image segmentation map, the saturation distribution map and the brightness distribution map comprises:

[0016] In units of image blocks of the image to be processed, the brightness and contrast of each image block are subjected to weight assignment in combination with the brightness distribution map, to obtain the image block brightness weight map, the image block contrast weight map and the exposure weight map of the image to be processed;

[0017] In combination with a set exposure threshold, pixel-level exposure weight calculation is performed on the saturation distribution map and the brightness distribution map to obtain the exposure weight map.

[0018] Preferably, the guided filtering and weighted fusion operation of the image block brightness weight map, the image block contrast weight map, the exposure weight map and the image to be processed to obtain a sequence base layer image and a sequence detail layer image comprise:

[0019] Exposure pixel points are selected according to the image block contrast weight map;

[0020] An exposure weight distribution map is obtained according to the exposure pixel points and the image block brightness weight map;

[0021] Based on the exposure weight distribution map, select pixels that meet the set conditions to form an optimal weight distribution map;

[0022] By combining different parameters of the guided filter, the preferred weight distribution map and the image to be processed are subjected to guided filtering to obtain the sequence base layer weight image and the sequence detail layer weight image.

[0023] Preferably, merging the sequence base layer image and the sequence detail layer image to obtain the fused image of the image to be processed, thereby enhancing the image to be processed, includes:

[0024] The image to be processed is layered by the guided filtering to obtain a layered image of the image to be processed;

[0025] Weight fusion is performed on the layered image, the sequence base layer weight image, and the sequence detail layer weight image to obtain the sequence base layer image and the sequence detail layer image;

[0026] The base layer image and the detail layer image of the sequence are merged to obtain the fused image.

[0027] Preferably, the step of processing the image to be processed in HSV format to obtain the saturation distribution map and brightness distribution map of the image to be processed includes:

[0028] The component saturation of the image to be processed is obtained based on the pixel value of each pixel in the image to be processed.

[0029] The tonal component of the image to be processed is obtained based on the maximum pixel value of all pixels in the image to be processed.

[0030] The maximum pixel value of all pixels in the image to be processed is assigned a brightness value to obtain the brightness component of the image to be processed.

[0031] Preferably, the step of assigning weights to the brightness and contrast of each image block, based on the image blocks of the image to be processed and in conjunction with the brightness distribution map, to obtain an image block brightness weight map and an image block contrast weight map of the image to be processed, includes:

[0032] Obtain the average brightness and contrast of each image block;

[0033] The maximum value among all the contrast ratios of the image blocks is taken to obtain the contrast weight;

[0034] By combining the two-dimensional Gaussian formula, the average brightness of all the image blocks is processed to obtain the brightness weight.

[0035] Secondly, this application provides an image enhancement processing system, the image enhancement processing system comprising:

[0036] Preprocessing unit: performs segmentation and HSV format processing on the image to be processed to obtain the image segmentation map, saturation distribution map and brightness distribution map of the image to be processed;

[0037] Image weighting unit: Based on the image segmentation map, the saturation distribution map, and the brightness distribution map, obtain the image block brightness weight map, image block contrast weight map, and exposure weight map of the image to be processed;

[0038] Image layering unit: Performs guided filtering and weighted fusion operations on the image block brightness weight map, the image block contrast weight map, the exposure weight map, and the image to be processed to obtain the sequence base layer image and the sequence detail layer image;

[0039] Image merging unit: merges the sequence base layer image and the sequence detail layer image to obtain a fused image of the image to be processed, thereby enhancing the image to be processed.

[0040] Meanwhile, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described method.

[0041] In addition, the present invention also provides a computer-readable storage medium storing a computer program for performing the above-described method.

[0042] As can be seen from the above technical solutions, the image enhancement processing method and system provided in this application integrates image patch-based weight calculation and pixel-based weight calculation, taking into account the contribution of the overall feature distribution of multi-exposure images as well as the contribution of local features or even individual pixels to the image. This provides a solution to the technical problem of the overall feature distribution or image patchiness in existing multi-exposure images. This invention features rich fused image information, clear details, and simple calculation, and can obtain high dynamic range images with uniform brightness and good exposure.

[0043] To make the above and other objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a flowchart illustrating an image enhancement processing method according to an embodiment of this application.

[0046] Figure 2 This is a schematic diagram of an image enhancement processing technique in an embodiment of this application.

[0047] Figure 3 This is a schematic diagram of the input of a hot air balloon multi-exposure image in an embodiment of this application.

[0048] Figure 4 This is a schematic diagram and implementation diagram of the superpixel segmentation algorithm in the embodiments of this application.

[0049] Figure 5 This is a schematic diagram of the image enhancement processing result in an embodiment of this application.

[0050] Figure 6 This is a schematic diagram of the structure of an image enhancement processing system according to an embodiment of this application.

[0051] Figure 7 This is a schematic diagram of the structure of the electronic device in the embodiments of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0053] Image patch-based methods are prone to block artifacts, leading to uneven brightness; while pixel-based methods focus more on local areas and fail to extract information from a holistic perspective. This application provides an image enhancement processing method, system, electronic device, and computer-readable storage medium that, when fusing image patch-based and pixel-based weights, employs a method that uses image patch-based brightness weights to supplement the fusion of exposure and contrast weights. This compensates for halo effects and other problems that may occur in pixel-based exposure weights.

[0054] The following examples and implementation details will be used to illustrate this further.

[0055] This application provides an embodiment of an image enhancement processing method, see [link to embodiment]. Figure 1 The image enhancement processing method specifically includes the following:

[0056] Step 100: Segment the image to be processed and process it in HSV format to obtain the image segmentation map, saturation distribution map and brightness distribution map of the image to be processed;

[0057] Step 200: Based on the image segmentation map, the saturation distribution map, and the brightness distribution map, obtain the image block brightness weight map, the image block contrast weight map, and the exposure weight map of the image to be processed;

[0058] Step 300: Perform guided filtering and weighted fusion operations on the image block brightness weight map, the image block contrast weight map, the exposure weight map, and the image to be processed to obtain the sequence base layer image and the sequence detail layer image;

[0059] Step 400: Merge the sequence base layer image and the sequence detail layer image to obtain a fused image of the image to be processed, thereby enhancing the image to be processed.

[0060] In this embodiment, the method of the present invention is implemented through the following technical solution:

[0061] Step (1): Input a multi-series exposure image in RGB format;

[0062] Step (2): Segment the multi-exposure images obtained in (1) to obtain image segmentation labels;

[0063] Step (3): Convert the overexposed image obtained in (1) into HSV format to obtain the saturation S distribution map and brightness V distribution map of the image to be processed.

[0064] Step (4): Combine the image segmentation map obtained in (2) and the brightness distribution map obtained in (3) to calculate the brightness weight of the image blocks and the contrast weight of the image blocks;

[0065] Step (5): Calculate pixel-level exposure weights using the saturation distribution map and brightness distribution map obtained in (3);

[0066] Step (6): Merge the weight maps obtained from (4) and (5), and perform guided filtering according to different parameters to obtain the sequence base layer weight map and the sequence detail layer weight map;

[0067] Step (7): Obtain the sequence base layer image and sequence detail layer image by applying fast guided filtering to the image obtained in (1);

[0068] Step (8): Combine the weights obtained from (6) and the images obtained from (7) to obtain a single detail layer image and a single base layer image, and merge them to obtain the final fused image with detail and uniform brightness.

[0069] As described above, the image enhancement processing method provided in this application combines image patch-based weight calculation and pixel-based weight calculation, taking into account the contribution of the overall feature distribution of multi-exposure images as well as the contribution of local features or even individual pixels to the image. This provides a solution to the technical problem of the overall feature distribution or image patchiness in existing multi-exposure images. This invention features rich fused image information, clear details, and simple calculation, and can obtain high dynamic range images with uniform brightness and good exposure.

[0070] In one embodiment of the image enhancement processing method provided in this application, the step of segmenting the image to be processed and processing it in HSV format to obtain an image segmentation map, a saturation distribution map, and a brightness distribution map of the image to be processed includes:

[0071] Superpixel segmentation is performed on the image to be processed to obtain an image segmentation map of the image to be processed;

[0072] The image to be processed is processed in HSV format to obtain the saturation distribution map and the brightness distribution map of the image to be processed.

[0073] In this embodiment, an image with relatively good exposure in step (1) is selected, and a segmentation map is obtained by uniform segmentation, non-uniform segmentation, or superpixel segmentation for all exposed images. The image segmentation map is then converted to a color space, which is an important parameter for calculating weights. The saturation S and brightness V in the HSV space are the main parameters required for the calculation.

[0074] Therefore, the pixel values ​​are converted to their corresponding color spaces to obtain the required values. The specific calculation is as follows: V = max(R,G,B), where R, G, and B are the red, yellow, and blue component pixel values ​​of pixel (i,j,k), respectively.

[0075] In one embodiment of the image enhancement processing method provided in this application, obtaining an image patch brightness weight map, an image patch contrast weight map, and an exposure weight map of the image to be processed based on the image segmentation map, the saturation distribution map, and the brightness distribution map includes:

[0076] Using the image blocks of the image to be processed as units, and combining the brightness distribution map, weight values ​​are assigned to the brightness and contrast of each image block to obtain the image block brightness weight map and the image block contrast weight map of the image to be processed.

[0077] By combining the set exposure threshold, pixel-level exposure weights are calculated on the saturation distribution map and the brightness distribution map to obtain the exposure weight map.

[0078] In this embodiment, for the sake of the overall image, the luminance weight and contrast weight are mainly calculated based on image blocks. That is, the average value of all luminance values ​​in an image block is the luminance component of all pixels in that image block. Similarly, the contrast component, which is the L2 norm of the luminance values ​​of all pixels in an image block minus the average value, is the contrast component of each pixel in that image block.

[0079] The image patch with the most information in a given image patch across different images has the highest contrast value. Therefore, this example selects the patch with the highest contrast value by comparing identical image patches across different images, assigning it a weight of 1, and setting the others to 0. The Gaussian formula can calculate appropriate exposure based on brightness, so the Gaussian formula is used to select the appropriate exposure weight for brightness. For global brightness weighting, this example uses a two-dimensional Gaussian formula, incorporating the image mean into the calculation.

[0080] When the values ​​of saturation (S) and brightness (V) are within the same range, the larger the absolute value of the difference between saturation (S) and brightness (V), the better the exposure of the corresponding pixel. In this example, 0.7 is chosen as the exposure threshold, which can better distinguish the appropriate level of exposure.

[0081] In one embodiment of the image enhancement processing method provided in this application, the step of performing guided filtering and weighted fusion operations on the image patch brightness weight map, the image patch contrast weight map, the exposure weight map, and the image to be processed to obtain a sequence of base layer images and a sequence of detail layer images includes:

[0082] Select the exposure pixels based on the image block contrast weight map;

[0083] Based on the exposure pixels and the image block brightness weight map, an exposure weight distribution map is obtained;

[0084] Based on the exposure weight distribution map, select pixels that meet the set conditions to form an optimal weight distribution map;

[0085] By combining different parameters of the guided filter, the preferred weight distribution map and the image to be processed are subjected to guided filtering to obtain the sequence base layer weight image and the sequence detail layer weight image.

[0086] In this embodiment, the exposure pixels are selected according to the contrast, and the exposure weight of the pixels is supplemented by the brightness weight of the image block to obtain a weight distribution map. The larger value is selected to obtain a weight distribution map with a value of (0,1). Then, different parameters of the guided filter are used to obtain weight distribution maps for the detail layer and the base layer respectively.

[0087] In step (6) above, the weights obtained in steps (4) and (5) above are first fused together. Then, the weight map obtained after fusion is processed by the principle of taking the largest value and then differentiated into two weight maps by guided filtering, namely the base layer weight map and the detail layer weight map. In order to differentiate, different parameters are selected as follows: detail layer parameters: r = 4, ε = 0.01; base layer parameters: r = 100, ε = 50.

[0088] Where r is the radius of the filtering window and ε is the regularization parameter; the formula for weight fusion is as follows:

[0089] W = ck_map.*sk_map.*lk_map;

[0090] Furthermore, in step (7) above, the parameters of the fast guided filtering in the method are selected as r = 12, ε = 0.25, and s = 3; the base layer is obtained through fast guided filtering, and the detail layer is obtained by subtracting the base layer from the source image; where r is the radius of the filtering window, ε is the regularization parameter, and s is the downsampling step size in fast guided filtering.

[0091] In one embodiment of the image enhancement processing method provided in this application, merging the sequence base layer image and the sequence detail layer image to obtain a fused image of the image to be processed, thereby enhancing the image to be processed, includes:

[0092] The image to be processed is layered by the guided filtering to obtain a layered image of the image to be processed;

[0093] Weight fusion is performed on the layered image, the sequence base layer weight image, and the sequence detail layer weight image to obtain the sequence base layer image and the sequence detail layer image;

[0094] The base layer image and the detail layer image of the sequence are merged to obtain the fused image.

[0095] In this embodiment, the fast guided filtering employed in this invention is an improvement upon guided filtering. By downsampling, it reduces the sampling and computation of pixels, thereby accelerating the filtering process. The source image is layered using fast guided filtering and then fused with weights to obtain the final image.

[0096] In one embodiment of the image enhancement processing method provided in this application, the step of performing HSV format processing on the image to be processed to obtain a saturation distribution map and a brightness distribution map of the image to be processed includes:

[0097] The component saturation of the image to be processed is obtained based on the pixel value of each pixel in the image to be processed.

[0098] The tonal component of the image to be processed is obtained based on the maximum pixel value of all pixels in the image to be processed.

[0099] The maximum pixel value of all pixels in the image to be processed is assigned a brightness value to obtain the brightness component of the image to be processed.

[0100] In this embodiment, the algorithm used in step (3) above to convert RGB format to HSV format has the following specific conversion steps:

[0101] Step (3.1): For the pixel value of a certain pixel, obtain the maximum and minimum values ​​of the three RGB components according to their magnitudes, and then subtract them, as follows:

[0102]

[0103] Step (3.2): Divide the difference obtained in step (3.1) by the maximum value to obtain the value of component saturation S, as follows:

[0104]

[0105] Step (3.3): Based on the maximum value obtained in step (3.1), determine the hue calculation and obtain the hue component H. The value range of H should be [0, 360]. When the result of H is less than 0°, the following applies to adding 360°:

[0106]

[0107] Step (3.4): Assign the maximum value obtained in step (3.1) to the luminance component V, as follows:

[0108] V = Cmax.

[0109] In one embodiment of the image enhancement processing method provided in this application, the step of assigning weights to the brightness and contrast of each image block, based on the image blocks of the image to be processed and combined with the brightness distribution map, to obtain an image block brightness weight map and an image block contrast weight map of the image to be processed, includes:

[0110] Obtain the average brightness and contrast of each image block;

[0111] The maximum value among all the contrast ratios of the image blocks is taken to obtain the contrast weight;

[0112] By combining the two-dimensional Gaussian formula, the average brightness of all the image blocks is processed to obtain the brightness weight.

[0113] In this embodiment, in step (4) above, the obtained image segmentation map and brightness distribution map are used to calculate the brightness weight and contrast weight of the image blocks. First, the pixels of the same image block are obtained through the image segmentation map labels, and the average brightness lk(n,k) of the nth image block in the kth image is calculated. At the same time, the contrast of the nth image block in the kth image is obtained by the L2 norm of the difference between the pixel brightness and the average brightness of the image block. Then, the contrast weight is obtained by taking the larger contrast value. The brightness weight is obtained by applying a two-dimensional Gaussian formula to the brightness, as follows:

[0114]

[0115]

[0116] ck_map(i,j,k)=(ck(n,k)==max(ck(n,:))),n=labels(i,j)+1

[0117]

[0118] The mean() function is the mean function. This refers to the brightness value matrix corresponding to the nth block of the kth image; (ck(n,k)==max(ck(n,:))) is a logical value calculation, which means that if ck is the maximum contrast value corresponding to the image block in the k images, then it takes the value 1, otherwise it is 0; uk(k) represents the overall brightness mean of the kth image.

[0119] The following is combined Figure 2 Taking four multi-exposure images of a hot air balloon as an example (see...) Figure 3 This application provides a detailed description of an image enhancement processing method:

[0120] (1) Image segmentation to obtain labels

[0121] The segmentation method is not limited; this example uses superpixel segmentation (see [link]). Figure 4 First, select an appropriately exposed image as a reference image, and convert the RGB color space image to a Lab color space image using XYZ conversion; then, select k cluster centers using a regular grid (interval step size s). in Initialize labels = 0; create element e and set Ck, k, and 0 as the values ​​of element e; then calculate the distance between the surrounding elements of each cluster and the cluster center pixel, using the following formula: Where s is the interval step size mentioned above, and m is the color normalization factor; and the value of C[k] is updated using the surrounding elements, so that the cluster center moves according to the distance and color gradient while continuously absorbing the surrounding pixels; thus, the final label map labels is obtained as the image segmentation map.

[0122] (2) Color space conversion (from RGB color space to HSV color space)

[0123] As important parameters for calculating weights, saturation S and brightness V in the HSV space are the main parameters required for calculation.

[0124] Therefore, the pixel values ​​are converted to their corresponding color spaces to obtain the required values. The specific calculation is as follows: V = max(R,G,B), where R, G, and B are the red, yellow, and blue component pixel values ​​of pixel (i,j,k), respectively.

[0125] (3) Calculation of brightness weight and contrast weight based on image patch

[0126] To maintain the integrity of the image, the luminance and contrast weights are calculated primarily based on image blocks. That is, the average luminance of all pixels in an image block is the luminance component of all pixels in that image block. Similarly, the contrast component is calculated as the L2 norm of the luminance of all pixels in an image block minus the average luminance.

[0127] The image patch with the most information in a given image patch across different images has the highest contrast value. Therefore, this example selects the patch with the highest contrast value by comparing identical image patches across different images, assigning it a weight of 1, and setting the others to 0. The Gaussian formula can calculate appropriate exposure based on brightness, so the Gaussian formula is used to select the appropriate exposure weight for brightness. For global brightness weighting, this example uses a two-dimensional Gaussian formula, incorporating the image mean into the calculation.

[0128] The specific formula is as follows:

[0129]

[0130]

[0131] ck_map(i,j,k)=(ck(n,k)==max(ck(n,:))),n=labels(i,j)+1

[0132]

[0133] The mean() function is the mean function. This refers to the brightness value matrix corresponding to the nth block of the kth image; (ck(n,k)==max(ck(n,:))) is a logical value calculation, which means that if ck is the maximum contrast value corresponding to the image block in the k images, then it takes the value 1, otherwise it is 0; uk(k) represents the overall brightness mean of the kth image.

[0134] (4) Pixel-level exposure weight calculation

[0135] When the values ​​of saturation (S) and brightness (V) are within the same range, the larger the absolute value of the difference between saturation (S) and brightness (V), the better the exposure of the corresponding pixel. In this example, 0.7 is chosen as the exposure threshold, which can better distinguish the appropriate level of exposure. The specific formula is as follows:

[0136] d(i,j,k)=|s(i,j,k)-v(i,j,k)|;

[0137] s(i,j,k)=1, if d(i,j,k)<0.7;

[0138] And then iterate d(i,j,k) again once;

[0139]

[0140] (5) Merge the weight graph and apply guided filtering.

[0141] Exposure pixels are selected based on contrast, and the exposure weights of these pixels are supplemented by the brightness weights of image blocks, resulting in a weight distribution map. A weight distribution map with values ​​of (0,1) is then selected based on the principle of taking the largest value. Finally, different parameters of the guided filter are used to obtain weight distribution maps for the detail layer and the base layer, respectively, as shown in the following formula:

[0142] w = ck_map.*sk_map.*lk_map

[0143] w(i,j,k)=(w(i,j,k)==max(w(i,j,:),3))

[0144] W D (:,:,1,:)=guidedfilter(v,w,4,0.01)

[0145] W B (:,:,:)=guidedfilter(v,w,100,50)

[0146] Here, `guidedfilter()` is the guided filter function, which filters the weight map `w` using the reference image `v` to obtain the relevant weight map. In this example, to facilitate the operation between `WD` and `D`, the matrix of `WD` is transformed into a 4-dimensional matrix, while the numerical values ​​remain unchanged.

[0147] (6) The fast guided filtering method used in this invention is an improvement on guided filtering. By downsampling, it reduces the sampling and computation of pixels, thus speeding up the filtering process. The source image is layered using fast guided filtering and then fused with weights to obtain the final image; the formula is as follows:

[0148] B(:,:,k)=fastguidedfilter(v(:,:,k),v(:,:,k),12,0.25,2),

[0149] D(i,j,:,k)=f(i,j,:,k)-B(i,j,k),

[0150] F = sum(D.*WD,4) + sum(B.*WB,3)

[0151] Where f(:,:,:,k) is the k-th source image, and in this example, k is 1-4; sum() is the summation function, and F is the final result image (see [link]). Figure 5 ).

[0152] Secondly, this application provides an embodiment of an image enhancement processing system, see [link to embodiment]. Figure 6 The image enhancement processing system specifically includes the following components:

[0153] Preprocessing unit 01: Segments and processes the image to be processed in HSV format to obtain the image segmentation map, saturation distribution map and brightness distribution map of the image to be processed;

[0154] Image weighting unit 02: Based on the image segmentation map, the saturation distribution map, and the brightness distribution map, obtain the image block brightness weight map, image block contrast weight map, and exposure weight map of the image to be processed;

[0155] Image layering unit 03: Performs guided filtering and weighted fusion operations on the image block brightness weight map, the image block contrast weight map, the exposure weight map, and the image to be processed to obtain the sequence base layer image and the sequence detail layer image;

[0156] Image merging unit 04: Merges the sequence base layer image and the sequence detail layer image to obtain a fused image of the image to be processed, thereby enhancing the image to be processed.

[0157] In this embodiment, the preprocessing unit 01 inputs a multi-sequence exposure image in RGB format; segments the multi-exposure image obtained in (1) to obtain image segmentation maps labels; and converts the multi-exposure image obtained in (1) into HSV format to obtain the saturation S distribution map and brightness V distribution map of the image to be processed.

[0158] Image weighting unit 02 calculates the image block brightness weight and image block contrast weight by combining the image segmentation map and brightness distribution map obtained by preprocessing unit 01; and performs pixel-level exposure weight calculation by using the saturation distribution map and brightness distribution map obtained by preprocessing unit 01.

[0159] Image layering unit 03 fuses the weight map obtained by image weighting unit 02, and performs guided filtering according to different parameters to obtain the sequence base layer weight map and the sequence detail layer weight map; the image obtained by preprocessing unit 01 is subjected to fast guided filtering to obtain the sequence base layer image and the sequence detail layer image.

[0160] Image merging unit 04 merges the weights and images obtained from image layering unit 03 to obtain a single detail layer image and a single base layer image, and then merges them to obtain a final fused image with detail and uniform brightness.

[0161] As described above, the image enhancement processing system provided in this application integrates image patch-based weight calculation and pixel-based weight calculation, taking into account the contribution of the overall feature distribution of multi-exposure images as well as the contribution of local features or even individual pixels to the image. This provides a solution to the technical problem of overall feature distribution or image patchiness in existing multi-exposure images. This invention features rich fused image information, clear details, and simple calculation, and can obtain high dynamic range images with uniform brightness and good exposure.

[0162] From a hardware perspective, image patch-based methods are prone to block artifacts, leading to uneven brightness; while pixel-based methods focus more on local areas and cannot extract information from a holistic perspective. This application provides an embodiment of an electronic device that incorporates all or part of the image enhancement processing method, specifically including the following:

[0163] Figure 7 This is a schematic block diagram illustrating the system configuration of the electronic device 9600 according to an embodiment of this application. Figure 7 As shown, the electronic device 9600 may include a central processing unit 9100 and a memory 9140; the memory 9140 is coupled to the central processing unit 9100. It is worth noting that... Figure 7 This is an example; other types of structures can also be used to supplement or replace this structure to achieve telecommunications functions or other functions.

[0164] In one embodiment, the image enhancement processing option can be integrated into a central processing unit (CPU). The CPU can be configured to perform the following control:

[0165] Step 100: Segment the image to be processed and process it in HSV format to obtain the image segmentation map, saturation distribution map and brightness distribution map of the image to be processed;

[0166] Step 200: Based on the image segmentation map, the saturation distribution map, and the brightness distribution map, obtain the image block brightness weight map, the image block contrast weight map, and the exposure weight map of the image to be processed;

[0167] Step 300: Perform guided filtering and weighted fusion operations on the image block brightness weight map, the image block contrast weight map, the exposure weight map, and the image to be processed to obtain the sequence base layer image and the sequence detail layer image;

[0168] Step 400: Merge the sequence base layer image and the sequence detail layer image to obtain a fused image of the image to be processed, thereby enhancing the image to be processed.

[0169] In this embodiment, the method of the present invention is implemented through the following technical solution:

[0170] Step (1): Input a multi-series exposure image in RGB format;

[0171] Step (2): Segment the multi-exposure images obtained in (1) to obtain image segmentation labels;

[0172] Step (3): Convert the overexposed image obtained in (1) into HSV format to obtain the saturation S distribution map and brightness V distribution map of the image to be processed.

[0173] Step (4): Combine the image segmentation map obtained in (2) and the brightness distribution map obtained in (3) to calculate the brightness weight of the image blocks and the contrast weight of the image blocks;

[0174] Step (5): Calculate pixel-level exposure weights using the saturation distribution map and brightness distribution map obtained in (3);

[0175] Step (6): Merge the weight maps obtained from (4) and (5), and perform guided filtering according to different parameters to obtain the sequence base layer weight map and the sequence detail layer weight map;

[0176] Step (7): Obtain the sequence base layer image and sequence detail layer image by applying fast guided filtering to the image obtained in (1);

[0177] Step (8): Combine the weights obtained from (6) and the images obtained from (7) to obtain a single detail layer image and a single base layer image, and merge them to obtain the final fused image with detail and uniform brightness.

[0178] As described above, the electronic device provided in this application integrates image patch-based weight calculation and pixel-based weight calculation, taking into account the contribution of the overall feature distribution of multi-exposure images as well as the contribution of local features or even individual pixels to the image. This provides a solution to the technical problem of the overall feature distribution or image patchwork effect in existing multi-exposure images. This invention features rich fused image information, clear details, and simple calculation, and can obtain high dynamic range images with uniform brightness and good exposure.

[0179] In another embodiment, the image enhancement processing device can be configured separately from the central processing unit 9100. For example, the image enhancement processing device can be configured as a chip connected to the central processing unit 9100, and the image enhancement processing function can be implemented through the control of the central processing unit.

[0180] like Figure 7 As shown, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily need to include these components. Figure 7 All components shown; in addition, the electronic device 9600 may also include Figure 7 For components not shown, please refer to existing technologies.

[0181] like Figure 7 As shown, the central processing unit 9100, sometimes also referred to as a controller or operating control, may include a microprocessor or other processor device and / or logic device, which receives inputs and controls the operation of various components of the electronic device 9600.

[0182] The memory 9140 may be, for example, one or more of a cache, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices. It may store the aforementioned failure-related information, and also store a program for executing that information. The central processing unit 9100 may execute the program stored in the memory 9140 to perform information storage or processing, etc.

[0183] Input unit 9120 provides input to central processing unit 9100. Input unit 9120 may be, for example, a keypad or touch input device. Power supply 9170 provides power to electronic device 9600. Display 9160 displays images and text. Display may be, for example, an LCD display, but is not limited thereto.

[0184] The memory 9140 can be a solid-state memory, such as a read-only memory (ROM), random access memory (RAM), a SIM card, etc. It can also be a memory that retains information even when power is off, can be selectively erased, and contains more data; examples of this type of memory are sometimes referred to as EPROMs. The memory 9140 can also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142 for storing application programs and function programs or processes for executing the operation of the electronic device 9600 via the central processing unit 9100.

[0185] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for the electronic device's communication functions and / or for performing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0186] The communication module 9110 is a transmitter / receiver 9110 that transmits and receives signals via the antenna 9111. The communication module (transmitter / receiver) 9110 is coupled to the central processing unit 9100 to provide input signals and receive output signals, which can be the same as in a conventional mobile communication terminal.

[0187] Based on different communication technologies, multiple communication modules 9110 can be configured in the same electronic device, such as cellular network modules, Bluetooth modules, and / or wireless LAN modules. The communication module (transmitter / receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and receive audio input from the microphone 9132, thereby realizing typical telecommunications functions. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. Additionally, the audio processor 9130 is coupled to a central processing unit 9100, enabling on-device recording via the microphone 9132 and on-device playback of stored sound via the speaker 9131.

[0188] Embodiments of this application also provide a computer-readable storage medium capable of implementing all steps of the image enhancement processing method in the above embodiments. The computer-readable storage medium stores a computer program that, when executed by a processor, implements all steps of the image enhancement processing method in the above embodiments where the execution subject is a server or client. For example, when the processor executes the computer program, it implements the following steps:

[0189] Step 100: Segment the image to be processed and process it in HSV format to obtain the image segmentation map, saturation distribution map and brightness distribution map of the image to be processed;

[0190] Step 200: Based on the image segmentation map, the saturation distribution map, and the brightness distribution map, obtain the image block brightness weight map, the image block contrast weight map, and the exposure weight map of the image to be processed;

[0191] Step 300: Perform guided filtering and weighted fusion operations on the image block brightness weight map, the image block contrast weight map, the exposure weight map, and the image to be processed to obtain the sequence base layer image and the sequence detail layer image;

[0192] Step 400: Merge the sequence base layer image and the sequence detail layer image to obtain a fused image of the image to be processed, thereby enhancing the image to be processed.

[0193] In this embodiment, the method of the present invention is implemented through the following technical solution:

[0194] Step (1): Input a multi-series exposure image in RGB format;

[0195] Step (2): Segment the multi-exposure images obtained in (1) to obtain image segmentation labels;

[0196] Step (3): Convert the overexposed image obtained in (1) into HSV format to obtain the saturation S distribution map and brightness V distribution map of the image to be processed.

[0197] Step (4): Combine the image segmentation map obtained in (2) and the brightness distribution map obtained in (3) to calculate the brightness weight of the image blocks and the contrast weight of the image blocks;

[0198] Step (5): Calculate pixel-level exposure weights using the saturation distribution map and brightness distribution map obtained in (3);

[0199] Step (6): Merge the weight maps obtained from (4) and (5), and perform guided filtering according to different parameters to obtain the sequence base layer weight map and the sequence detail layer weight map;

[0200] Step (7): Obtain the sequence base layer image and sequence detail layer image by applying fast guided filtering to the image obtained in (1);

[0201] Step (8): Combine the weights obtained from (6) and the images obtained from (7) to obtain a single detail layer image and a single base layer image, and merge them to obtain the final fused image with detail and uniform brightness.

[0202] As described above, the computer-readable storage medium provided in this application integrates image patch-based weight calculation and pixel-based weight calculation, taking into account the contribution of the overall feature distribution of multi-exposure images as well as the contribution of local features or even individual pixels to the image. This provides a solution to the technical problem of overall feature distribution or image patchiness in existing multi-exposure images. This invention features rich fused image information, clear details, and simple calculation, and can obtain high dynamic range images with uniform brightness and good exposure.

[0203] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0204] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0205] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0206] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 Figure 1 The steps of the function specified in one or more boxes.

[0207] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. An image enhancement processing method characterized by, The method comprises the following steps: segmenting and processing the image in HSV format to obtain an image segmentation map, a saturation distribution map and a brightness distribution map of the image to be processed; obtaining an image block brightness weight map, an image block contrast weight map and an exposure weight map of the image to be processed according to the image segmentation map, the saturation distribution map and the brightness distribution map; performing guided filtering and weighted fusion operation on the image block brightness weight map, the image block contrast weight map, the exposure weight map and the image to be processed to obtain a sequence base layer image and a sequence detail layer image; merging the sequence base layer image and the sequence detail layer image to obtain a fusion image of the image to be processed, and then enhancing the image to be processed; the guided filtering and weighted fusion operation on the image block brightness weight map, the image block contrast weight map, the exposure weight map and the image to be processed to obtain a sequence base layer image and a sequence detail layer image comprises: selecting an exposure pixel point according to the image block contrast weight map; obtaining an exposure weight distribution map according to the exposure pixel point and the image block brightness weight map; selecting a pixel point meeting a set condition to form an optimal weight distribution map according to the exposure weight distribution map; performing guided filtering on the optimal weight distribution map and the image to be processed in combination with different parameters of guided filtering to obtain a sequence base layer weight image and a sequence detail layer weight image; the obtaining of an image block brightness weight map, an image block contrast weight map and an exposure weight map of the image to be processed according to the image segmentation map, the saturation distribution map and the brightness distribution map comprises: performing weight assignment on the brightness and contrast of each image block in combination with the brightness distribution map to obtain an image block brightness weight map and an image block contrast weight map of the image to be processed; performing pixel-level exposure weight calculation on the saturation distribution map and the brightness distribution map in combination with a set exposure threshold to obtain the exposure weight map.

2. The image enhancement processing method according to claim 1, characterized by, the segmentation and HSV format processing of the image to be processed to obtain an image segmentation map, a saturation distribution map and a brightness distribution map of the image to be processed comprises: performing superpixel segmentation on the image to be processed to obtain an image segmentation map of the image to be processed; performing HSV format processing on the image to be processed to obtain a saturation distribution map and a brightness distribution map of the image to be processed.

3. The image enhancement processing method of claim 1, wherein, the merging of the sequence base layer image and the sequence detail layer image to obtain a fusion image of the image to be processed, and then enhancing the image to be processed comprises: performing layering on the image to be processed through the guided filtering to obtain a layered image of the image to be processed; performing weight fusion on the layered image, the sequence base layer weight image and the sequence detail layer weight image to obtain the sequence base layer image and the sequence detail layer image; merging the sequence base layer image and the sequence detail layer image to obtain the fusion image.

4. The image enhancement processing method of claim 2, wherein, The HSV format processing of the to-be-processed image obtains a saturation distribution map and a brightness distribution map of the to-be-processed image, and comprises: According to the pixel value of each pixel point of the to-be-processed image, the component saturation of the to-be-processed image is obtained; According to the maximum pixel value of all pixel points of the to-be-processed image, the hue component of the to-be-processed image is obtained; The maximum pixel value of all pixel points of the to-be-processed image is assigned with brightness, and the brightness component of the to-be-processed image is obtained.

5. The image enhancement processing method of claim 1, wherein, The image block of the to-be-processed image is taken as a unit, the brightness and contrast of each image block are assigned with weights in combination with the brightness distribution map, and the image block brightness weight map and the image block contrast weight map of the to-be-processed image are obtained, comprising: The average brightness and contrast of each image block are obtained; The maximum value of the contrast of all image blocks is taken, and the contrast weight is obtained; In combination with the two-dimensional Gaussian formula, the average brightness of all image blocks is processed, and the brightness weight is obtained.

6. An image enhancement processing system characterized by comprising: Comprise: The preprocessing unit: the to-be-processed image is segmented and processed in HSV format, and the image segmentation map, the saturation distribution map and the brightness distribution map of the to-be-processed image are obtained; The image weight unit: according to the image segmentation map, the saturation distribution map and the brightness distribution map, the image block brightness weight map, the image block contrast weight map and the exposure weight map of the to-be-processed image are obtained; the image weight unit also takes the image block of the to-be-processed image as a unit, assigns the brightness and contrast of each image block with weights in combination with the brightness distribution map, and obtains the image block brightness weight map and the image block contrast weight map of the to-be-processed image; In combination with the set exposure threshold, the pixel level exposure weight calculation of the saturation distribution map and the brightness distribution map is performed, and the exposure weight map is obtained; The image layering unit: the image block brightness weight map, the image block contrast weight map, the exposure weight map and the to-be-processed image are guided filtering and weighted fusion operation, and the sequence base layer image and the sequence detail layer image are obtained; The image merging unit: the sequence base layer image and the sequence detail layer image are merged, and the fusion image of the to-be-processed image is obtained, so that the to-be-processed image is enhanced; The image layering unit comprises: The first subunit is used for selecting exposure pixel points according to the image block contrast weight map; The second subunit is used for obtaining an exposure weight distribution map according to the exposure pixel points and the image block brightness weight map; The third subunit is used for selecting pixel points meeting the set conditions to form an optimal weight distribution map according to the exposure weight distribution map; The fourth subunit is used for guiding filtering processing of the optimal weight distribution map and the to-be-processed image in combination with different parameters of the guided filtering, and obtaining the sequence base layer weight image and the sequence detail layer weight image.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to realize the steps of the image enhancement processing method in any one of claims 1 to 5.

8. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to realize the steps of the image enhancement processing method in any one of claims 1 to 5.

Images