Image enhancement method and device, equipment and storage medium

Abstract

The invention provides an image enhancement method and device, equipment and a storage medium. The method comprises the following steps: dividing an image to be processed into a plurality of image blocks, and determining first tone mapping information, first low-frequency information and high-frequency information of the image blocks; performing tone mapping on the first low-frequency information of the image block according to the first tone mapping information of the image block to obtain second low-frequency information; fusing the second low-frequency information and the high-frequency information of the image block at least according to the texture feature, the first brightness feature and the second brightness feature of the image block to obtain fusion information of the image block; and determining an enhanced image of the to-be-processed image according to the respective fusion information of the plurality of image blocks.

Description

Technical Field

[0001] The present application relates to image technology, and is related to but not limited to image enhancement methods and devices, equipment, and storage media. Background Art

[0002] In the field of image processing, image contrast enhancement is an important technical means to improve image visual effects / image quality, and can be applied to the post-processing stage or other image analysis and processing stages of images taken by electronic devices. Although some image contrast enhancement algorithms can improve the image contrast to a certain extent, there is still an unnatural brightness transition at the edge of the image after image enhancement. Summary of the invention

[0003] In view of this, the image enhancement method, device, equipment, and storage medium provided in the present application can make the edge brightness of the enhanced image more natural and reduce the occurrence of brightness flipping, halo, and other phenomena.

[0004] According to one aspect of an embodiment of the present application, there is provided an image enhancement method, the method comprising: dividing an image to be processed into a plurality of image blocks, determining first tone mapping information and first low-frequency information and high-frequency information of the image blocks; performing tone mapping on the first low-frequency information of the image blocks according to the first tone mapping information of the image blocks to obtain second low-frequency information; fusing the second low-frequency information and high-frequency information of the image blocks according to at least texture features, first brightness features and second brightness features of the image blocks to obtain fusion information of the image blocks; and determining an enhanced image of the image to be processed according to the fusion information of each of the plurality of image blocks.

[0005] According to one aspect of an embodiment of the present application, an image enhancement device is provided, comprising: a division module, configured to divide an image to be processed into multiple image blocks; a first determination module, configured to determine first tone mapping information and first low-frequency information and high-frequency information of the image block; a mapping module, configured to perform tone mapping on the first low-frequency information of the image block according to the first tone mapping information of the image block to obtain second low-frequency information; a fusion module, configured to fuse the second low-frequency information and high-frequency information of the image block according to at least texture features, first brightness features, and second brightness features of the image block to obtain fusion information of the image block; and a second determination module, configured to determine an enhanced image of the image to be processed according to the fusion information of each of the multiple image blocks.

[0006] According to one aspect of an embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the method described in the embodiment of the present application is implemented.

[0007] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method provided in the embodiment of the present application is implemented.

[0008] In an embodiment of the present application, when the low-frequency information and the high-frequency information of an image block are fused, the texture features and the image content of the image block are taken into consideration; that is, the low-frequency information and the high-frequency information of the image block are fused according to the texture features and the brightness features of the image block, rather than fusion of the two according to a fixed preset fusion weight; this is beneficial for better enhancing the image details and avoiding or reducing the unnatural edge brightness transition or even brightness flipping, halo and other phenomena in the obtained enhanced image.

[0009] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0011] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0012] Figure 1 This is a schematic diagram of brightness reversal and halo;

[0013] Figure 2 Schematic diagram of the implementation process of the image enhancement method provided in the embodiment of the present application Figure 1 ;

[0014] Figure 3 A schematic diagram of a flow chart of a method for determining first tone mapping information provided in an embodiment of the present application;

[0015] Figure 4 A schematic diagram of the implementation flow of step 302 provided in the embodiment of the present application;

[0016] Figure 5 A schematic diagram of the implementation flow of step 203 provided in the embodiment of the present application;

[0017] Figure 6 Schematic diagram of neighboring blocks of the current block provided in the embodiment of the present application Figure 1 ;

[0018] Figure 7 Schematic diagram of neighboring blocks of the current block provided in the embodiment of the present application Figure 2 ;

[0019] Figure 8 Schematic diagram of weights of neighboring blocks of the current block provided in the embodiment of the present application Figure 1 ;

[0020] Fig. 9 A schematic diagram of a flow chart of a method for determining fusion information of an image block provided in an embodiment of the present application;

[0021] Fig.10 A schematic diagram of the implementation flow of step 2052 provided in the embodiment of the present application;

[0022] Fig.11 A schematic diagram of the implementation flow of step 1003 provided in the embodiment of the present application;

[0023] Fig.12 A schematic diagram of the bilinear interpolation principle provided in an embodiment of the present application;

[0024] Fig.13 A schematic diagram of the implementation flow of step 1102 provided in an embodiment of the present application;

[0025] Fig.14 Schematic diagram of the implementation process of the image enhancement method provided in the embodiment of the present application Figure 2 ;

[0026] Fig.15 Schematic diagram of the implementation process of the image enhancement method provided in the embodiment of the present application Figure 3 ;

[0027] Fig.16 A schematic diagram of image segmentation and texture detection provided in an embodiment of the present application;

[0028] Fig.17 A schematic diagram of the fusion of the Log mapping curve and the S curve provided in an embodiment of the present application;

[0029] Fig.18 A schematic diagram of using a tone mapping curve of a neighborhood block for smoothing provided in an embodiment of the present application;

[0030] Fig.19 A schematic diagram of the structure of an image enhancement device provided in an embodiment of the present application;

[0031] Fig. 20 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the specific technical solution of the present application will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but are not used to limit the scope of the present application.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0034] In the following description, reference is made to “some embodiments”, “this embodiment”, “embodiments of the present application” and examples, etc., which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0035] The descriptions of “first, second, third” etc. that appear in the embodiments of the present application are only for illustration and distinction of the description objects. There is no distinction of order, nor do they indicate any special limitation on the number of devices in the embodiments of the present application, and cannot constitute any limitation on the embodiments of the present application.

[0036] The tone mapping methods of images are mainly divided into two categories, one is a tone mapping method based on histogram, and the other is a tone mapping method based on frequency division. The embodiment of the present application improves the tone mapping method based on frequency division. First, the tone mapping scheme based on frequency division is explained, which mainly includes the following three steps 1) to 3): 1) First, the image is layered into different high-frequency information and low-frequency information; 2) The low-frequency information is adjusted using curve mapping to adjust its brightness and contrast; 3) The adjusted low-frequency information is fused with the high-frequency information to retain and enhance the details of the image. Among them, step 1) mainly uses an edge-preserving smoothing filtering algorithm, such as a weighted least squares filtering algorithm, a bilateral filtering algorithm, and the like. Step 2) can use various brightness adjustment algorithms. Including Log-based curves, S-shaped curves, and algorithms based on histogram adjustment can also be used in this step to adjust the brightness and contrast. The fusion step in step 3) is mainly based on artificial adjustment of fusion parameters in related schemes, and there are few algorithms with good effects in this step.

[0037] Through the analysis of the existing frequency division-based tone mapping scheme, it is found that the existing frequency division-based tone mapping scheme fails to fully utilize the image content and does not fully consider the local information of the image. First, if the current low-frequency information and high-frequency information are not fused with reasonable fusion weights, the brightness of the image edge will be unnatural and even brightness reversal, halo and other phenomena will occur. For example, Figure 1 From the enhanced image shown, taking the image content in the rectangular frame 101 as an example, it can be seen that there is a brightness reversal around the lights on the branches, that is, the lights on the branches should not be so dark, but should be brighter, with a natural transition, and should not suddenly turn black; secondly, the current mainstream algorithms do not consider adjusting the brightness, contrast and details of the face separately, while the face is the focus of people's attention in practical applications.

[0038] Based on the above analysis, an embodiment of the present application provides an image enhancement method, which is applied to an electronic device. During the implementation process, the electronic device can be various types of devices with image processing capabilities, for example, the electronic device can include a mobile phone, a tablet computer, a desktop computer, a drone, a laptop computer, etc. The function implemented by the method can be implemented by calling a program code by a processor in the electronic device. Of course, the program code can be stored in a computer storage medium. It can be seen that the electronic device at least includes a processor and a storage medium.

[0039] Figure 2 Schematic diagram of the implementation process of the image enhancement method provided in the embodiment of the present application Figure 1 ,like Figure 2 As shown, the method may include the following steps 201 to 206:

[0040] Step 201, dividing the image to be processed into a plurality of image blocks;

[0041] Step 202, determining first tone mapping information of the image block;

[0042] Step 203, determining first low-frequency information and high-frequency information of the image block;

[0043] Step 204, performing tone mapping on the first low-frequency information of the image block according to the first tone mapping information of the image block to obtain second low-frequency information;

[0044] Step 205, fusing the second low-frequency information and the high-frequency information of the image block at least according to the texture feature, the first brightness feature and the second brightness feature of the image block to obtain fused information of the image block;

[0045] Step 206: Determine an enhanced image of the image to be processed according to the fusion information of each of the multiple image blocks.

[0046] In an embodiment of the present application, when the low-frequency information and the high-frequency information of an image block are fused, the texture features and the image content of the image block are taken into consideration; that is, the low-frequency information and the high-frequency information of the image block are fused according to the texture features and the brightness features of the image block, rather than fusion of the two according to a fixed preset fusion weight; this is beneficial for better enhancing the image details and avoiding or reducing the unnatural edge brightness transition or even brightness flipping, halo and other phenomena in the obtained enhanced image.

[0047] The following describes further optional implementations and related terms of each of the above steps.

[0048] In step 201, the image to be processed is divided into a plurality of image blocks.

[0049] The electronic device may divide the image to be processed into a plurality of image blocks of the same size or different sizes. For example, the image to be processed may be divided into a plurality of square blocks or rectangular blocks of the same size, or the image to be processed may be divided into image blocks of irregular shapes. In the embodiment of the present application, there is no limitation on the way of dividing the image blocks.

[0050] In step 202, first tone mapping information of the image block is determined.

[0051] In the embodiment of the present application, the tone mapping information can be understood as a tone mapping curve (Tone Mapping curve, TM curve), and can also be understood as brightness mapping information / curve or grayscale mapping information / curve.

[0052] In the embodiment of the present application, there is no limitation on the method for determining the first tone mapping information, and there may be various methods. In short, as long as the contrast enhancement effect of the image block can be achieved.

[0053] In some embodiments, the electronic device may determine the first tone mapping information as follows: the first tone mapping information is determined based on the texture features (such as the texture pixel ratio) and the second brightness features (such as the brightness mean) of the image block; in this way, the texture features and the brightness features are used as the basis for determining the first tone mapping information, so that the determined first tone mapping information is adapted to the image content of the image block, thereby benefiting to better enhance the subjective visual effect of the enhanced image.

[0054] In the embodiments of the present application, for different image blocks, the method of determining the first tone mapping information may be the same or different. For example, for an image block belonging to a face area, the electronic device may determine the first tone mapping information of the image block by using the following embodiment 1, or may determine the first tone mapping information of the image block by using the following embodiment 2. For an image block in a non-face area, the electronic device may determine the first tone mapping information of the image block by using the following embodiment 2.

[0055] In Example 1, the electronic device may determine the first tone mapping information as follows: when the image block belongs to a face area, based on the texture features and the second brightness features of the image block, query a preset face tone mapping information library to obtain the first tone mapping information; in this way, when performing image enhancement on the processed image, consideration is given to adjusting the brightness, contrast and details of the face alone, which is beneficial to better control the light and shadow effects of the face area.

[0056] It should be noted that the so-called texture feature of an image block refers to information that can reflect the texture characteristics / features / characteristics of the image block. For example, the texture feature includes the proportion of texture pixels in the image block. The so-called brightness feature of an image block refers to information that can reflect the brightness characteristics / features / characteristics of the image block. For example, the second brightness feature of an image block includes the brightness mean of the image block.

[0057] In Example 2, Figure 3 As shown, the electronic device may also determine the first tone mapping information through the following steps 301 and 302:

[0058] Step 301: Determine second tone mapping information and third tone mapping information of the image block.

[0059] It can be understood that the second tone mapping information and the third tone mapping information are mapping curves determined by different methods. For example, for the second tone mapping information, in some embodiments, the electronic device can determine the second tone mapping information based on the first brightness feature of the image block and the third brightness feature of the image to be processed.

[0060] Furthermore, in some embodiments, the electronic device may determine a Log mapping curve (ie, an example of the second tone mapping information) of the image block according to the first brightness feature of the image block and the third brightness feature of the image to be processed.

[0061] It can be understood that the so-called brightness feature of the image to be processed refers to information that can reflect the overall brightness characteristics / features / characteristics of the image. For example, the third brightness feature of the image to be processed includes the maximum brightness value of the image. The first brightness feature of the image block includes the maximum brightness value and the minimum brightness value of the image block.

[0062] Specifically, for example, the electronic device determines the Log mapping curve f2(v) of the image block according to the following formula:

[0063]

[0064] Where v represents the brightness value, v = 0, 1, 2, ... Max; Max refers to the maximum brightness value in the image to be processed, β1 is a setting parameter, (i, j) represents the block index of the image block, V max (i, j) represents the maximum brightness value of the image block (i, j), V min (i,j) represents the minimum brightness value of image block (i,j).

[0065] Of course, in other embodiments, the second tone mapping information of the image block may also be determined by a grayscale histogram-based method, for example, by using a contrast limited adaptive histogram equalization (CLAHE) algorithm to determine the second tone mapping information of the image block.

[0066] In some embodiments, the third tone mapping information may be obtained based on global information (i.e., information of the entire image to be processed). For example, a global S-shaped curve may be preset as the third tone mapping information of the image block. The S-shaped curves of each image block may be the same or different.

[0067] Step 302: According to the texture feature (such as the texture pixel ratio) and the second brightness feature (such as the brightness mean) of the image block, the brightness mapping values ​​of the same brightness level in the second tone mapping information and the third tone mapping information are merged to obtain the first tone mapping information.

[0068] It can be understood that in step 302, when the second tone mapping information and the third tone mapping information are fused, not only the texture features of the image block but also the image content of the image block (i.e., the second brightness feature) are considered; compared with fusing the two according to a pre-set fixed fusion weight, the first tone mapping information that is more in line with the actual content of the image block can be obtained, so that the edge brightness transition of the enhanced image obtained based on the tone mapping information is more natural, thereby obtaining an enhanced image with better subjective visual effect.

[0069] Furthermore, in some embodiments, Figure 4 As shown, the electronic device can implement step 302 through the following steps 3031 and 3032:

[0070] Step 3021: Determine a fifth fusion weight according to the texture feature and the second brightness feature of the image block.

[0071] For example, in some embodiments, the electronic device may determine the fifth fusion weight fw according to the following formula:

[0072] fw=(a1*t r (i,j)+b1)*c1*V mean (i,j)+d1

[0073] Among them, a1, b1, c1 and d1 are setting parameters, t r (i, j) is the texture pixel ratio of the image block with block index (i, j), V mean (i, j) is the mean brightness value of the image block with block index (i, j).

[0074] Step 3022: According to the fifth fusion weight, the brightness mapping values ​​of the same brightness level in the second tone mapping information and the third tone mapping information are fused to obtain the first tone mapping information.

[0075] In an embodiment of the present application, the fifth fusion weight can be used as the fusion weight of the second tone mapping information, and can also be used as the fusion weight of the third tone mapping information. When the fifth fusion weight is used as the fusion weight of the second tone mapping information, 1 minus the fifth fusion weight is used as the fusion weight of the third tone mapping information. Vice versa, when the fifth fusion weight is used as the fusion weight of the third tone mapping information, 1 minus the fifth fusion weight is used as the fusion weight of the second tone mapping information.

[0076] Assuming that the second tone mapping information is represented by f2(v), the third tone mapping information is represented by f3(v), the first tone mapping information is represented by f1(v), and the fifth fusion weight is represented by fw, in an example, f1(v) can be fused as follows:

[0077] f1(v)=(1-fw)*f2(v)+fw*f3(v)

[0078] In another example, f1(v) can be fused like this:

[0079] f1(v)=fw*f2(v)+(1-fw)*f3(v)

[0080] In step 203, first low-frequency information and high-frequency information of the image block are determined.

[0081] In the embodiment of the present application, the low-frequency information of the image block can also be understood as the low-frequency signal, low-frequency image, basic information or basic layer information of the image block, etc. The high-frequency information of the image block can also be understood as the high-frequency information, high-frequency image, detail information or detail layer information of the image block.

[0082] In some embodiments, Figure 5 As shown, the electronic device can implement step 203 through the following steps 2031 to 2033:

[0083] Step 2031 : performing at least one edge-preserving smoothing filtering process on the image to be processed to obtain a low-frequency image of the image to be processed.

[0084] In the embodiment of the present application, the low-frequency image of the image to be processed can also be understood as the low-frequency information, low-frequency signal, basic information or basic layer information of the image.

[0085] In some embodiments, the electronic device may perform one edge-preserving smoothing filtering process on the image to be processed, or may perform multiple iterative edge-preserving smoothing filtering processes. The iterative edge-preserving smoothing filtering process refers to further performing edge-preserving smoothing filtering process on the low-frequency image currently processed, and the low-frequency image in step 2031 refers to the low-frequency image finally output after all the sub-edge-preserving smoothing filtering processes are performed on the image to be processed.

[0086] In the embodiment of the present application, the filtering algorithm used in the edge-preserving smoothing filtering process is not limited, and may be any type of filtering algorithm capable of decomposing the base layer information, for example, a weighted least squares filtering algorithm or a bilateral filtering algorithm.

[0087] Step 2032: Determine a high-frequency image of the image to be processed according to the image to be processed and the low-frequency image.

[0088] In the embodiment of the present application, the high-frequency image of the image to be processed can also be understood as the high-frequency information, high-frequency signal, detail information or detail layer information of the image.

[0089] As described above for step 2031, the electronic device may perform one edge-preserving smoothing filtering process on the image to be processed, or may perform multiple iterations of edge-preserving smoothing filtering process. For performing only one edge-preserving smoothing filtering process on the image to be processed, the high-frequency image obtained in step 2032 is one image, and the electronic device may subtract the low-frequency image from the image to be processed, thereby obtaining a high-frequency image of the image to be processed.

[0090] For performing multiple iterations of edge-preserving smoothing filtering on the image to be processed, the high-frequency images obtained in step 2032 are two images. Taking two times as an example, the electronic device performs one edge-preserving smoothing filtering on the image to be processed to obtain a low-frequency image 1, and then further performs the edge-preserving smoothing filtering on the low-frequency image 2. Figure 1 For example, an edge-preserving smoothing filtering process is performed once to obtain a low-frequency image 2; the electronic device subtracts the low-frequency image 1 from the image to be processed to obtain a high-frequency image 1; the electronic device subtracts the low-frequency image 2 from the image to be processed to obtain a high-frequency image 2; wherein the low-frequency image 2 is an example of the low-frequency image described in step 2031, and the high-frequency image 1 and the high-frequency image 2 are examples of the high-frequency image described in step 2032. Similarly, for performing iterative edge-preserving smoothing filtering processes more than twice, the high-frequency image described in step 2032 may include a high-frequency image obtained based on the low-frequency image output by each edge-preserving smoothing filtering process.

[0091] Step 2033: obtain first low-frequency information of the image block based on the low-frequency image, and obtain high-frequency information of the image block based on the high-frequency image.

[0092] It can be understood that if the region information / position information of the image block is known, the first low-frequency information and high-frequency information of the corresponding position can be found from the low-frequency image and the high-frequency image.

[0093] In step 204, tone mapping is performed on the first low-frequency information of the image block according to the first tone mapping information of the image block to obtain second low-frequency information.

[0094] In an embodiment of the present application, the electronic device can use the first tone mapping information to directly perform tone mapping on the first low-frequency information of the corresponding image block, or it can first optimize the first tone mapping information (such as the smoothing processing mentioned below), and then use the optimized first tone mapping information to perform tone mapping.

[0095] Specifically, in some embodiments, the electronic device may implement step 204 as follows: using the first tone mapping information of the neighborhood block of the image block to smooth the first tone mapping information of the image block to obtain fourth tone mapping information of the image block; and using the fourth tone mapping information to tone map the first low-frequency information to obtain the second low-frequency information.

[0096] Exemplarily, in some embodiments, the electronic device may determine the fourth tone mapping information tmf of the image block according to the following formula: final [v]:

[0097] Wherein, (i, j) refers to the block index of the image block, which here refers to the block index of the current block (i.e., the image block described in step 204) and the block index of the neighboring blocks of the current block. i,j It refers to the smoothing weight of the image block with block index (i, j).

[0098] In the embodiment of the present application, the neighborhood block is not limited and may include one or more image blocks directly adjacent to the current block, for example, Figure 6 As shown, assuming that rectangular block 601 is the current block, the neighboring blocks of the current block include rectangular blocks 602 to 609; the neighboring blocks may also include one or more image blocks indirectly adjacent to the current block, for example, Figure 7 As shown, the neighboring blocks of the current block include rectangular blocks 602 to 609 and rectangular blocks 701 to 716.

[0099] In the embodiment of the present application, different neighborhood blocks use smoothing weights w i,j They can be the same or different. When they are different, the size of the smoothing weight of the neighboring block is negatively correlated with the distance from the current block. The farther away from the current block, the smaller the corresponding smoothing weight; the closer to the current block, the larger the corresponding smoothing weight. The smoothing weight is distributed in a Gaussian kernel. For example, Figure 8 As shown, the smoothing weight w0 of the current block>the smoothing weight w1 of the neighboring block>the smoothing weight w2 of the neighboring block>the smoothing weight w3 of the neighboring block>the smoothing weight w4 of the neighboring block>the smoothing weight w5 of the neighboring block.

[0100] In step 205, the second low-frequency information and the high-frequency information of the image block are fused at least according to the texture feature, the first brightness feature and the second brightness feature of the image block to obtain fused information of the image block.

[0101] In some embodiments, the electronic device may implement step 205 as follows: according to the texture feature, the first brightness feature and the second brightness feature of the image block, the second low-frequency information and the high-frequency information of the image block are merged.

[0102] In other embodiments, the electronic device may also implement step 205 as follows: according to the texture features, the first brightness features, the second brightness features and the face size of the image block, and the third brightness features and the fourth brightness features of the image to be processed, the second low-frequency information and the high-frequency information of the image block are fused to obtain the fused information of the image block; in this way, when the second low-frequency information and the high-frequency information are fused, the parameter of the face size is also considered, which is beneficial to enhance the details of the face area, and thus enhance the subjective visual effect of the face area.

[0103] Furthermore, in some embodiments, Fig. 9As shown, the fusion information of the image block can be obtained through the following steps 2051 and 2052:

[0104] Step 2051: Determine the brightness weighted contrast of the image block according to the first brightness feature, the second brightness feature, the third brightness feature, and the fourth brightness feature.

[0105] Exemplarily, in some embodiments, the first brightness feature of the image block includes a maximum brightness value and a minimum brightness value, the second brightness feature of the image block includes a brightness mean, the third brightness feature of the image to be processed includes a maximum brightness value, and the fourth brightness feature of the image to be processed includes a minimum brightness value.

[0106] For example, the electronic device may determine the brightness weighted contrast Crt of the image block according to the following formula:

[0107]

[0108] Among them, V mean Represents the mean brightness of the image block, I lmax Represents the maximum brightness value of the image block, I lmin Represents the minimum brightness value of the image block, V mean Represents the mean brightness of the image block, I max Indicates the maximum brightness value of the image to be processed, I min represents the minimum brightness value of the image to be processed, k1, β2 are setting parameters, V mean Represents the mean brightness of the image block.

[0109] Step 2052: According to the brightness weighted contrast, the texture feature of the image block and the face size, the second low-frequency information and the high-frequency information of the image block are fused to obtain fused information of the image block.

[0110] Furthermore, in some embodiments, Fig.10 As shown, step 2052 can be implemented through the following steps 1001 to 1003:

[0111] Step 1001: Determine a first fusion weight of the second low-frequency information according to the brightness weighted contrast and the texture feature of the image block.

[0112] Exemplarily, in some embodiments, the texture feature of the image block includes a texture pixel ratio.

[0113] The electronic device can determine the first fusion weight w of the second low-frequency information according to the following formula: b :

[0114]

[0115] Among them, t r is the texture pixel ratio of the image block, Crt is the brightness weighted contrast of the image block, and a2, b2, c2, d2 are set parameters.

[0116] Step 1002: Determine a second fusion weight of the high-frequency information according to the brightness weighted contrast, the texture feature of the image block, and the face size.

[0117] The electronic device can determine the second fusion weight w of the high-frequency information according to the following formula:

[0118]

[0119] Among them, t r is the texture pixel ratio of the image block, Crt is the brightness weighted contrast of the image block, and a3, b3, c3, d3, e, k2 are the setting parameters. face size is the size of the face. size It represents the proportion of face pixels or the number of face pixels in the image block.

[0120] Step 1003: According to the first fusion weight and the second fusion weight, the second low-frequency information and the high-frequency information of the image block are fused to obtain fusion information of the image block.

[0121] In an embodiment of the present application, the electronic device may directly fuse the second low-frequency information and the high-frequency information of the image block according to the first fusion weight and the second fusion weight, or may first optimize the second low-frequency information of the image block (for example, the bilinear interpolation mentioned below), and then fuse the optimized second low-frequency information and the high-frequency information.

[0122] Specifically, in some embodiments, Fig.11 As shown, step 1003 can be implemented by the following steps 1101 and 1102:

[0123] Step 1101: Perform bilinear interpolation on the second low-frequency information of the image block to obtain third low-frequency information.

[0124] In some embodiments, step 1101 may be implemented as follows: using the fourth tone mapping information of the neighborhood block of the image block, bilinearly interpolate the second low-frequency information of the image block to obtain the third low-frequency information.

[0125] It can be understood that since each image block has different fourth tone mapping information, it will cause block effects, so a bilinear interpolation method can be used to eliminate the block effects. Fig.12As shown, for pixel O, assuming that its gray value is r0, for pixel O, the gray value is r0, and the fourth tone mapping information tmf of the four nearest blocks around it final Record tmf separately A ,tmf B ,tmf C ,tmf D , then the result of bilinear interpolation s o As shown below:

[0126] s o =(1-x)*(1-y)*tmf A (r0)+x*(1-y)*tmf B (r0)

[0127] +(1-x)*y*tmf c (r0)+x*y*tmf D (r0)

[0128] Step 1102: According to the first fusion weight and the second fusion weight, the third low-frequency information and the high-frequency information are fused to obtain fusion information of the image block.

[0129] In an embodiment of the present application, the electronic device can directly fuse the third low-frequency information and the high-frequency information based on the first fusion weight and the second fusion weight, or it can first optimize the first fusion weight and the second fusion weight respectively (such as the smoothing process below), and then fuse the third low-frequency information and the high-frequency information based on the optimized fusion weight.

[0130] Specifically, in some embodiments, Fig.13 As shown, the electronic device can implement step 1102 through the following steps 1301 to 1303:

[0131] Step 1301 : using the first fusion weight of the second low-frequency information of the neighborhood block of the image block, smoothing the first fusion weight of the second low-frequency information of the image block to obtain a third fusion weight.

[0132] The neighborhood blocks here may be the same as or different from the neighborhood blocks mentioned above, and the neighborhood blocks used to smooth the first fusion weight of the current block may include part or all of the image blocks directly adjacent to the current block, and may also include part or all of the image blocks indirectly adjacent to the current block.

[0133] Step 1302: Use the second fusion weight of the high frequency information of the neighborhood block of the image block to smooth the second fusion weight of the high frequency information of the image block to obtain a fourth fusion weight.

[0134] The neighborhood blocks here may be the same as or different from the neighborhood blocks mentioned above, and the neighborhood blocks used to smooth the second fusion weights of the current block may include part or all of the image blocks directly adjacent to the current block, and may also include part or all of the image blocks indirectly adjacent to the current block.

[0135] Step 1303: fuse the third low-frequency information and the high-frequency information according to the third fusion weight and the fourth fusion weight to obtain fusion information of the image block.

[0136] As described above, the electronic device may perform edge-preserving smoothing filtering processing only once on the image to be processed, or may perform multiple iterative edge-preserving smoothing filtering processing on the image to be processed. Therefore, the high-frequency information of the image block may include one high-frequency image block, or may include multiple high-frequency image blocks. Taking the example of performing two iterative edge-preserving smoothing filtering processing on the image to be processed, the high-frequency information of the image block described in step 1303 includes the high-frequency image block I obtained based on the high-frequency image 1 described above. detail1 , and the high-frequency image block I obtained based on the high-frequency image 2 as described above detail2 Assume that the third low-frequency information of the image block is represented as I base , I detail1 The fourth fusion weight is denoted as w1, I detail2 The fourth fusion weight is represented as w2, I base The third fusion weight is expressed as w3, and the electronic device can determine the fusion information of the image block according to the following formula:

[0137] I fus =w3*I base +w1*I detail1 +w2*I detail2

[0138] In step 206, an enhanced image of the image to be processed is determined according to the fusion information of each of the plurality of image blocks.

[0139] In an embodiment of the present application, the electronic device can directly use the fusion information of each of the multiple image blocks as the image content of the enhanced image, or it can first optimize the fusion information of each of the multiple image blocks (such as the bilinear interpolation mentioned below), and then use the optimized fusion information as the image content of the enhanced image.

[0140] Specifically, in some embodiments, step 206 may be implemented as follows: bilinear interpolation is performed on the fusion information of each of the plurality of image blocks to obtain the enhanced image.

[0141] The present application embodiment further provides an image enhancement method, Fig.14Schematic diagram of the implementation process of the image enhancement method provided in the embodiment of the present application Figure 2 ,like Fig.14 As shown, the method includes the following steps 1401 to 1416:

[0142] Step 1401, dividing the image to be processed into a plurality of image blocks;

[0143] Step 1402, determining second tone mapping information of the image block according to the first brightness feature of the image block and the third brightness feature of the image to be processed;

[0144] Step 1403, obtaining third tone mapping information that is preset based on global information;

[0145] Step 1404, determining a fifth fusion weight according to the texture feature and the second brightness feature of the image block;

[0146] Step 1405: According to the fifth fusion weight, the brightness mapping values ​​of the same brightness level in the second tone mapping information and the third tone mapping information are fused to obtain first tone mapping information of the image block;

[0147] Step 1406, performing at least one edge-preserving smoothing filtering process on the image to be processed to obtain a low-frequency image of the image to be processed;

[0148] Step 1407, determining a high-frequency image of the image to be processed according to the image to be processed and the low-frequency image;

[0149] Step 1408, obtaining first low-frequency information of the image block based on the low-frequency image, and obtaining high-frequency information of the image block based on the high-frequency image;

[0150] It should be noted that, for the sake of simplicity, step 1408 is not shown in the figure.

[0151] Step 1409, using the first tone mapping information of the neighborhood block of the image block, smoothing the first tone mapping information of the image block to obtain fourth tone mapping information of the image block;

[0152] Step 1410, performing tone mapping on the first low-frequency information using the fourth tone mapping information to obtain the second low-frequency information;

[0153] Step 1411, determining a brightness weighted contrast of the image block according to the first brightness feature and the second brightness feature of the image block and the third brightness feature and the fourth brightness feature of the image to be processed;

[0154] Step 1412, determining a first fusion weight of the second low-frequency information according to the brightness weighted contrast and the texture feature of the image block;

[0155] Step 1413, determining a second fusion weight of the high-frequency information according to the brightness weighted contrast, the texture feature of the image block and the face size;

[0156] Step 1414, performing bilinear interpolation on the second low-frequency information of the image block to obtain third low-frequency information;

[0157] Step 1415, fusing the third low-frequency information and the high-frequency information according to the first fusion weight and the second fusion weight to obtain fusion information of the image block;

[0158] Step 1416: perform bilinear interpolation on the fusion information of each of the multiple image blocks to obtain an enhanced image of the image to be processed.

[0159] The following describes an exemplary application of the embodiments of the present application in a practical application scenario.

[0160] In an embodiment of the present application, an image enhancement algorithm is provided. First, the scheme divides the image to be processed into blocks and obtains the texture information, contrast and statistical information of each block. And calculate the tone mapping curve of each block. At the same time, the edge-preserving filtering algorithm (i.e., edge-preserving filtering algorithm) is used to decompose the image to be processed into a base layer and a detail layer, and the base layer image is further decomposed using the edge-preserving filtering algorithm again (this scheme uses two decompositions as an example, and the actual use does not limit the number of decompositions). For each block of the final base layer, the scheme uses the calculated tone mapping curve to perform tone mapping. The base layer that has been tone mapped is reconstructed using texture information, and the reconstruction method is to weighted superposition of the base layer image and the detail layer. In the tone mapping and reconstruction process, this scheme will additionally use facial information as a reference.

[0161] The specific flow chart is as follows Fig.15 As shown, combined Fig.15 The details of this plan are as follows:

[0162] Step 1: Divide the image to be processed into blocks and perform texture detection on the image to be processed.

[0163] For example, in some embodiments, the Sobel algorithm may be used to perform texture detection on the processed image, where the convolution factor d in the Sobel X-axis direction is x Convolution factor d with the Y axis y As shown in the following formula:

[0164] The Sobel convolution factor is used to calculate the image A to be processed, and the gradient value G in the X-axis direction is obtained respectively. x The gradient value G in the Y-axis direction y , the specific calculation is shown in the following formula:

[0165] G x =d x *A and G y =d y *A

[0166] The gradient value of the image is positively correlated with the texture. The gradient value is calculated by the gradient values ​​in the X-axis and Y-axis directions. The calculation formula is as follows:

[0167]

[0168] For example, Fig.16 As shown, the image on the left is a schematic diagram of the result of dividing the image to be processed into blocks, and the image on the right is a schematic diagram of the texture image obtained after texture detection is performed on the image to be processed.

[0169] After obtaining the texture image, this scheme counts the texture weight of each image block of the texture image. The ratio of the texture pixels in the current block to all the pixels in the current block is calculated according to the following formula to obtain the texture pixel ratio of the current block t r :

[0170]

[0171] Among them, Num texture Indicates the number of texture pixels in the image block, Num total Represents the total number of pixels in the image block.

[0172] Step 2. Calculate the tone mapping curve of each image block of the image to be processed. The specific calculation method is as follows:

[0173] (2.1) Traverse each image block and calculate the maximum brightness, minimum brightness, and average brightness in each image block, which are denoted as V max (i,j),V min (i,j),V mean (i,j); i,j are the coordinate indexes of each block, that is, the block index.

[0174] (2.2) For each image block in the processed image, a Log mapping curve is generated according to the maximum brightness and the minimum brightness obtained in (2.1). The specific curve generation formula is as follows:

[0175]

[0176] Where Max is the maximum brightness value in the image to be processed, β1 is the set parameter, and the value range of the brightness variable v is as follows:

[0177] v=0,1,2,…Max

[0178] (2.3) Artificially set a global S-shaped curve (hereinafter referred to as S curve). And according to the image content, the S curve is fused with the Log mapping curve generated in (2.2). The fusion weight is determined by the brightness mean and texture information of the image block. The more texture areas there are, the greater the weight of the S curve is, and the smaller the brightness mean of the image block is, the greater the weight of the Log mapping curve is. The weight calculation formula of the S curve is as follows:

[0179] fw=(a1*t r (i,j)+b1)*c1*V mean (i,j)+d1

[0180] Among them, a1, b1, c1 and d1 are setting parameters, t r (i, j) is the texture pixel ratio of the image block with block index (i, j), V mean (i,j) is the mean brightness of the image block with block index (i,j). The weight of the Log curve is 1-fw.

[0181] For example, Fig.17 As shown, the tone mapping curve 171 is an example of a Log mapping curve, the tone mapping curve 172 is an example of an S curve, and the tone mapping curve 173 is a fusion result of the curve 171 and the curve 172 .

[0182] (2.4) For the face area, the electronic device queries the face tone mapping curve library according to the texture pixel ratio of the current block and the brightness mean of the current block to find the corresponding curve (ie, the first tone mapping information).

[0183] Step 3. Calculate the brightness weighted contrast of each image block of the image to be processed. The calculation formula of the brightness weighted contrast is as follows:

[0184]

[0185] Among them I lmax Indicates the maximum brightness in the image block, I lmin Indicates the minimum brightness in the image block. max Indicates the maximum brightness of the entire image to be processed, I min Indicates the minimum brightness in the entire image to be processed, V mean Represents the mean brightness of the image block, k1, β2 are set parameters.

[0186] Step 4: Perform edge-preserving smoothing on the graphics to be processed.

[0187] Use edge-preserving smoothing filtering algorithms (such as weighted least squares filtering or bilateral filtering) to perform edge-preserving smoothing filtering on the image to be processed. After using the edge-preserving smoothing filtering algorithm, a low-frequency base layer image with details removed is obtained. The image of the detail layer can be obtained by subtracting the image of the low-frequency base layer from the image to be processed. The base layer retains the basic image information of the image, and the detail layer retains the detail information of the image.

[0188] Step 5. Perform tone mapping on the resulting base layer image.

[0189] For each pixel in the base layer image, the tone mapping operation can be completed by directly calculating the mapped value according to the tone mapping curve calculated in step 2.

[0190] At the same time, in order to remove the block effect, it is necessary to smooth the mapping curves and mapped values ​​of different image blocks. First, the tone mapping curve is smoothed. For example, Fig.18 As shown, for the tone mapping curve in the image block 181, the tone mapping curve of the neighboring block is used for smoothing.

[0191] The specific smoothing formula is as follows:

[0192]

[0193] where tmf final [v] is the tone mapping curve after smoothing of the current block, tmf i,j [v] is Fig.18 The tone mapping curve corresponding to the neighborhood block in the box 182 is shown as w i,j is the smoothing weight of the corresponding block, exemplarily, as Fig.18 As shown. The value range of brightness variable v is as follows:

[0194] v=0,1,2,…Max

[0195] After completing the above curve smoothing, when performing tone mapping, this scheme performs bilinear interpolation on the final mapping value of each pixel according to the neighborhood information to obtain the final mapping pixel value. Since each image block has a different mapping function, it will cause block effects, so bilinear interpolation is needed to eliminate block effects.

[0196] Step 6. Fusion of base layer and detail layer:

[0197] The final image is reconstructed by weighted fusion of the mapped base layer obtained in step 5 and the detail layer obtained in step 4. The fusion curve contains texture information, contrast information and face information. The calculation formula for the fusion weight of the detail layer is as follows:

[0198]

[0199] Among them, a3, b3, c3, d3, e, k2 are setting parameters, face size Indicates the size of the face. From the formula, we can see that the detail layer weight is small and the contrast is low in the area with high contrast and low texture, and the detail layer weight is large in the area with more texture. If there is a face area, the weight of the detail layer is appropriately increased to enhance the details of the face area.

[0200] Since the face information has been used in the tone mapping stage, the fusion weight of the base layer no longer considers the face information. Similar to the fusion area weight of the detail layer, the calculation method is as follows:

[0201]

[0202] Among them, t r is the texture pixel ratio of the image block, Crt is the brightness weighted contrast of the image block, and a2, b2, c2, d2 are set parameters.

[0203] After obtaining the weights of all layers, this scheme reconstructs the image according to the following formula:

[0204] I rec =w b *I base +w4*I detail1 +w5*I detail2

[0205] It should be noted that the calculation method of weight w4 is the same as that of weight w5, but their parameter values ​​are different.

[0206] In order to prevent the block effect, the weight w can be b , w4, w5 are smoothed, and the reconstructed image is determined using the weights after smoothing. The smoothing method is similar to the smoothing method in step 5. First, the smoothing weights are set for the fusion weights in different image blocks, and then the fusion weights are smoothed according to the smoothing weights. In the process of pixel-by-pixel reconstruction, the bilinear interpolation method can also be used to obtain the final fusion weight of each pixel.

[0207] The embodiment of the present application provides an image enhancement method, which can better perform tone mapping on an image according to the image content, that is, the tone mapping curve of a local area can be determined by using texture information and image adaptively, and the light and shadow effects of the face area can be better controlled, and the detail intensity of the local area can be controlled according to the texture information and the image content. Therefore, the subjective visual effect of the image can be better improved by using this solution.

[0208] It is understandable that in the embodiments of the present application, related user data such as images are involved. When the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0209] It should be noted that although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.; or, steps in different embodiments may be combined into a new technical solution.

[0210] Based on the foregoing embodiments, an embodiment of the present application provides an image enhancement device, which includes the modules included and the units included in the modules, which can be implemented by a processor; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be an image signal processing (Image Signal Processing, ISP), an AI acceleration engine (such as NPU, etc.), a GPU, a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc.

[0211] Fig.19 A schematic diagram of the structure of the image enhancement device provided in the embodiment of the present application is shown in FIG. Fig.19 As shown, the image enhancement device 190 includes:

[0212] A division module 1901 is configured to divide the image to be processed into a plurality of image blocks;

[0213] A first determination module 1902 is configured to determine first tone mapping information and first low-frequency information and high-frequency information of the image block;

[0214] A mapping module 1903 is configured to perform tone mapping on the first low-frequency information of the image block according to the first tone mapping information of the image block to obtain second low-frequency information;

[0215] A fusion module 1904 is configured to fuse the second low-frequency information and the high-frequency information of the image block at least according to the texture feature, the first brightness feature and the second brightness feature of the image block to obtain fusion information of the image block;

[0216] The second determination module 1905 is configured to determine an enhanced image of the image to be processed according to the fusion information of each of the multiple image blocks.

[0217] In some embodiments, the fusion module 1904 is configured to: fuse the second low-frequency information and the high-frequency information of the image block according to the texture features, the first brightness features, the second brightness features and the face size of the image block, and the third brightness features and the fourth brightness features of the image to be processed, to obtain the fusion information of the image block.

[0218] Furthermore, in some embodiments, the fusion module 1904 is configured to: determine the brightness weighted contrast of the image block based on the first brightness feature and the second brightness feature and the third brightness feature and the fourth brightness feature; and fuse the second low-frequency information and the high-frequency information of the image block based on the brightness weighted contrast, the texture feature of the image block and the face size to obtain the fusion information of the image block.

[0219] Furthermore, in some embodiments, the fusion module 1904 is configured to: determine a first fusion weight of the second low-frequency information based on the brightness weighted contrast and the texture features of the image block; and determine a second fusion weight of the high-frequency information based on the brightness weighted contrast, the texture features of the image block and the face size; and fuse the second low-frequency information and the high-frequency information of the image block based on the first fusion weight and the second fusion weight to obtain the fusion information of the image block.

[0220] Exemplarily, in some embodiments, the texture feature of the image block includes a texture pixel ratio, the first brightness feature of the image block includes a maximum brightness value and a minimum brightness value, the second brightness feature of the image block includes a brightness mean, the third brightness feature of the image to be processed includes a maximum brightness value, and the fourth brightness feature of the image to be processed includes a minimum brightness value.

[0221] Furthermore, in some embodiments, the fusion module 1904 is configured to: perform bilinear interpolation on the second low-frequency information of the image block to obtain third low-frequency information; and fuse the third low-frequency information with the high-frequency information according to the first fusion weight and the second fusion weight to obtain fusion information of the image block.

[0222] Exemplarily, in some embodiments, the fusion module 1904 is configured to: use the first fusion weight of the second low-frequency information of the neighborhood block of the image block to smooth the first fusion weight of the second low-frequency information of the image block to obtain a third fusion weight; use the second fusion weight of the high-frequency information of the neighborhood block of the image block to smooth the second fusion weight of the high-frequency information of the image block to obtain a fourth fusion weight; and fuse the third low-frequency information with the high-frequency information according to the third fusion weight and the fourth fusion weight to obtain the fusion information of the image block.

[0223] In some embodiments, the second determination module 1905 is configured to: perform bilinear interpolation on the fusion information of each of the multiple image blocks to obtain the enhanced image.

[0224] In some embodiments, the first determination module 1902 is configured to determine the first tone mapping information according to the texture feature and the second brightness feature of the image block.

[0225] Further, in some embodiments, the first determination module 1902 is configured to: determine the second tone mapping information and the third tone mapping information of the image block; and merge the brightness mapping values ​​of the same brightness level in the second tone mapping information and the third tone mapping information according to the texture features and the second brightness features of the image block to obtain the first tone mapping information.

[0226] Furthermore, in some embodiments, the first determination module 1902 is configured to determine the second tone mapping information according to the first brightness feature of the image block and the third brightness feature of the image to be processed.

[0227] Furthermore, in some embodiments, the first determination module 1902 is configured to: determine a fifth fusion weight based on the texture features and the second brightness features of the image block; and fuse the brightness mapping values ​​of the same brightness level in the second tone mapping information and the third tone mapping information according to the fifth fusion weight to obtain the first tone mapping information.

[0228] Further, in some embodiments, in some embodiments, the first determination module 1902 is configured to: when the image block belongs to a face area, query a preset face tone mapping information library according to the texture features and the second brightness features of the image block to obtain the first tone mapping information.

[0229] In some embodiments, the first determination module 1902 is configured to: perform at least one edge-preserving smoothing filtering process on the image to be processed to obtain a low-frequency image of the image to be processed; determine a high-frequency image of the image to be processed based on the image to be processed and the low-frequency image; obtain first low-frequency information of the image block based on the low-frequency image, and obtain high-frequency information of the image block based on the high-frequency image.

[0230] In some embodiments, the mapping module 1903 is configured to: use the first tone mapping information of the neighborhood block of the image block to smooth the first tone mapping information of the image block to obtain fourth tone mapping information of the image block; and use the fourth tone mapping information to tone map the first low-frequency information to obtain the second low-frequency information.

[0231] Furthermore, in some embodiments, the fusion module 1904 is configured to: perform bilinear interpolation on the second low-frequency information of the image block using the fourth tone mapping information of the neighborhood block of the image block to obtain the third low-frequency information.

[0232] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0233] It should be noted that the division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit. It may also be implemented in the form of a combination of software and hardware.

[0234] It should be noted that in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable an electronic device to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0235] An embodiment of the present application provides an electronic device, Fig. 20 Schematic diagram of the hardware entity of the electronic device of the embodiment of the present application, such as Fig. 20 As shown, the electronic device 20 includes a memory 2001 and a processor 2002. The memory 2001 stores a computer program that can be run on the processor 2002. When the processor 2002 executes the program, the steps in the method provided in the above embodiment are implemented.

[0236] It should be noted that the memory 2001 is configured to store instructions and applications executable by the processor 2002, and can also cache data to be processed or already processed by the processor 2002 and various modules in the electronic device 20 (for example, image data, audio data, voice communication data, and video communication data), which can be implemented through flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0237] An embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the method provided in the above embodiment are implemented.

[0238] An embodiment of the present application provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute the steps of the method provided in the above method embodiment.

[0239] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium, storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0240] It should be understood that "one embodiment" or "an embodiment" or "some embodiments" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in one embodiment" or "in some embodiments" appearing throughout the specification may not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments. The above description of each embodiment tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. For the sake of brevity, this article will not repeat them.

[0241] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist at the same time, and object B exists alone.

[0242] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0243] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as: multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or modules can be electrical, mechanical or other forms.

[0244] The modules described above as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules; they may be located in one place or distributed on multiple network units; some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0245] In addition, all functional modules in the embodiments of the present application may be integrated into one processing unit, or each module may be a separate unit, or two or more modules may be integrated into one unit; the above-mentioned integrated modules may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0246] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0247] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling an electronic device to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0248] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0249] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0250] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0251] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image enhancement method, characterized in that: The method comprises: Dividing the image to be processed into a plurality of image blocks, and determining first tone mapping information and first low-frequency information and high-frequency information of the image blocks; Performing tone mapping on the first low-frequency information of the image block according to the first tone mapping information of the image block to obtain second low-frequency information; fusing the second low-frequency information and the high-frequency information of the image block at least according to the texture feature, the first brightness feature, and the second brightness feature of the image block to obtain fused information of the image block; An enhanced image of the image to be processed is determined according to the fusion information of each of the multiple image blocks.

2. The method according to claim 1, characterized in that The fusing the second low-frequency information and the high-frequency information of the image block at least according to the texture feature, the first brightness feature and the second brightness feature of the image block to obtain the fused information of the image block includes: According to the texture feature, the first brightness feature, the second brightness feature and the face size of the image block, and the third brightness feature and the fourth brightness feature of the image to be processed, the second low-frequency information and the high-frequency information of the image block are fused to obtain the fused information of the image block.

3. The method according to claim 2, characterized in that The step of fusing the second low-frequency information and the high-frequency information of the image block according to the texture feature, the first brightness feature, the second brightness feature and the face size of the image block, and the third brightness feature and the fourth brightness feature of the image to be processed to obtain the fused information of the image block includes: Determining a brightness weighted contrast of the image block according to the first brightness feature, the second brightness feature, the third brightness feature, and the fourth brightness feature; The second low-frequency information and the high-frequency information of the image block are fused according to the brightness weighted contrast, the texture feature of the image block and the face size to obtain fused information of the image block.

4. The method according to claim 3, characterized in that The fusing the second low-frequency information and the high-frequency information of the image block according to the brightness weighted contrast, the texture feature of the image block and the face size to obtain the fused information of the image block includes: determining a first fusion weight of the second low-frequency information according to the brightness weighted contrast and the texture feature of the image block; Determining a second fusion weight of the high-frequency information according to the brightness weighted contrast, the texture feature of the image block and the size of the face; The second low-frequency information and the high-frequency information of the image block are fused according to the first fusion weight and the second fusion weight to obtain fusion information of the image block.

5. The method according to claim 3 or 4, characterized in that: The texture feature of the image block includes the texture pixel ratio of the image block, the first brightness feature of the image block includes the maximum brightness value of the image block and the minimum brightness value of the image block, the second brightness feature of the image block includes the brightness mean of the image block, the third brightness feature of the image to be processed includes the maximum brightness value of the image to be processed, and the fourth brightness feature of the image to be processed includes the minimum brightness value of the image to be processed.

6. The method according to claim 4, characterized in that The fusing the second low-frequency information and the high-frequency information of the image block according to the first fusion weight and the second fusion weight to obtain the fusion information of the image block includes: Performing bilinear interpolation on the second low-frequency information of the image block to obtain third low-frequency information; The third low-frequency information and the high-frequency information are fused according to the first fusion weight and the second fusion weight to obtain fusion information of the image block.

7. The method according to claim 6, characterized in that The fusing the third low-frequency information and the high-frequency information according to the first fusion weight and the second fusion weight to obtain the fusion information of the image block includes: Using the first fusion weight of the second low-frequency information of the neighborhood block of the image block, smoothing the first fusion weight of the second low-frequency information of the image block to obtain a third fusion weight; Using the second fusion weight of the high frequency information of the neighborhood block of the image block, smoothing the second fusion weight of the high frequency information of the image block to obtain a fourth fusion weight; The third low-frequency information and the high-frequency information are fused according to the third fusion weight and the fourth fusion weight to obtain fusion information of the image block.

8. The method according to claim 1, characterized in that The step of determining the enhanced image of the image to be processed according to the fusion information of each of the plurality of image blocks comprises: Bilinear interpolation is performed on the fusion information of each of the multiple image blocks to obtain the enhanced image.

9. The method according to any one of claims 1 to 8, characterized in that: Determining first tone mapping information of the image block includes: The first tone mapping information is determined according to the texture feature and the second brightness feature of the image block.

10. The method according to claim 9, characterized in that The determining the first tone mapping information according to the texture feature and the second brightness feature of the image block includes: Determining second tone mapping information and third tone mapping information of the image block; According to the texture feature and the second brightness feature of the image block, brightness mapping values ​​of the same brightness level in the second tone mapping information and the third tone mapping information are merged to obtain the first tone mapping information.

11. The method according to claim 10, characterized in that Determining second tone mapping information of the image block includes: The second tone mapping information is determined according to the first brightness feature of the image block and the third brightness feature of the image to be processed.

12. The method according to claim 10, characterized in that The step of fusing the brightness mapping values ​​of the same brightness level in the second tone mapping information and the third tone mapping information according to the texture feature and the second brightness feature of the image block to obtain the first tone mapping information includes: Determining a fifth fusion weight according to the texture feature and the second brightness feature of the image block; According to the fifth fusion weight, the brightness mapping values ​​of the same brightness level in the second tone mapping information and the third tone mapping information are fused to obtain the first tone mapping information.

13. The method according to claim 9, characterized in that The determining the first tone mapping information according to the texture feature and the second brightness feature of the image block includes: In the case that the image block belongs to a face area, a preset face tone mapping information library is queried according to the texture feature and the second brightness feature of the image block to obtain the first tone mapping information.

14. The method according to any one of claims 1 to 13, characterized in that: Determining first low-frequency information and high-frequency information of the image block includes: Performing at least one edge-preserving smoothing filtering process on the image to be processed to obtain a low-frequency image of the image to be processed; Determining a high-frequency image of the image to be processed according to the image to be processed and the low-frequency image; First low-frequency information of the image block is obtained based on the low-frequency image, and high-frequency information of the image block is obtained based on the high-frequency image.

15. The method according to claim 6, characterized in that The step of performing tone mapping on the first low-frequency information of the image block according to the first tone mapping information of the image block to obtain second low-frequency information includes: Using the first tone mapping information of the neighborhood block of the image block, smoothing the first tone mapping information of the image block to obtain fourth tone mapping information of the image block; The first low-frequency information is tone mapped by using the fourth tone mapping information to obtain the second low-frequency information.

16. The method according to claim 15, characterized in that The performing bilinear interpolation on the second low-frequency information of the image block to obtain third low-frequency information includes: The third low-frequency information is obtained by performing bilinear interpolation on the second low-frequency information of the image block by using the fourth tone mapping information of the neighborhood block of the image block.

17. An image enhancement device, characterized in that: The device comprises: A division module, configured to divide the image to be processed into a plurality of image blocks; A first determining module, configured to determine first tone mapping information and first low-frequency information and high-frequency information of the image block; a mapping module configured to perform tone mapping on the first low-frequency information of the image block according to the first tone mapping information of the image block to obtain second low-frequency information; a fusion module configured to fuse the second low-frequency information and the high-frequency information of the image block at least according to the texture feature, the first brightness feature and the second brightness feature of the image block to obtain fusion information of the image block; The second determination module is configured to determine an enhanced image of the image to be processed according to the fusion information of each of the multiple image blocks.

18. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 16 is implemented.

19. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 16 is implemented.

Images