Image processing method and device, computer readable storage medium and electronic device

Abstract

The present disclosure provides an image processing method and device, a computer readable storage medium and an electronic device, and relates to the technical field of image processing. The image processing method comprises: acquiring a to-be-processed image, and dividing the to-be-processed image into a plurality of image blocks; determining a gray histogram of each image block, and determining a histogram mapping curve of each image block according to the gray histogram of each image block; determining the gray maximum and minimum values of each image block, and determining a logarithmic domain mapping curve of each image block by using the gray maximum and minimum values of each image block; determining a fusion mapping curve of each image block according to the histogram mapping curve and the logarithmic domain mapping curve of each image block; and performing image enhancement on the to-be-processed image by using the fusion mapping curve of each image block. The present disclosure can improve the image quality.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and more specifically, to an image processing method, an image processing apparatus, a computer-readable storage medium, and an electronic device. Background Technology

[0002] In the field of image processing technology, image contrast enhancement is an important technique for improving the visual effect of images. It can be applied to the post-processing stage of images captured by electronic devices or other image analysis and processing stages.

[0003] Currently, some image contrast enhancement algorithms can improve image contrast to a certain extent; however, the quality of the processed image may still be poor after implementing these algorithms. Summary of the Invention

[0004] This disclosure provides an image processing method, an image processing apparatus, a computer-readable storage medium, and an electronic device, thereby overcoming, at least to some extent, the problem of poor image quality after image enhancement processing.

[0005] According to a first aspect of this disclosure, an image processing method is provided, comprising: acquiring an image to be processed; dividing the image to be processed into multiple image blocks; determining a gray-level histogram for each image block; determining a histogram mapping curve for each image block based on the gray-level histogram; determining the gray-level extreme values ​​for each image block; determining a logarithmic domain mapping curve for each image block using the gray-level extreme values; determining a fusion mapping curve for each image block based on the histogram mapping curve and the logarithmic domain mapping curve; and performing image enhancement on the image to be processed using the fusion mapping curve of each image block.

[0006] According to a second aspect of this disclosure, an image processing apparatus is provided, comprising: an image segmentation module for acquiring an image to be processed and dividing the image to be processed into multiple image blocks; a first curve determination module for determining a grayscale histogram of each image block and determining a histogram mapping curve of each image block based on the grayscale histogram of each image block; a second curve determination module for determining the grayscale extreme values ​​of each image block and determining a logarithmic domain mapping curve of each image block using the grayscale extreme values ​​of each image block; a third curve determination module for determining a fusion mapping curve of each image block based on the histogram mapping curve and the logarithmic domain mapping curve of each image block; and an image enhancement module for performing image enhancement on the image to be processed using the fusion mapping curve of each image block.

[0007] According to a third aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the image processing method described above.

[0008] According to a fourth aspect of this disclosure, an electronic device is provided, including a processor; and a memory for storing one or more programs, which, when executed by the processor, cause the processor to perform the image processing method described above.

[0009] In some embodiments of this disclosure, the histogram mapping curve of each image block in the image to be processed is determined, the logarithmic domain mapping curve of each image block is determined based on the maximum and minimum gray values ​​of each image block, the fusion mapping curve of the image block is determined based on the histogram mapping curve and the logarithmic domain mapping curve, and the fusion mapping curve of each image block is used to enhance the image to be processed. On one hand, this disclosure utilizes the logarithmic domain mapping curve determined based on the maximum and minimum gray values ​​of the image blocks for image enhancement. The use of the maximum and minimum gray values ​​of the image blocks and the logarithmic domain transformation method can improve the local contrast of the image and enhance image quality. On the other hand, the image block mapping curve applied in the image enhancement step of this disclosure is obtained by fusing the histogram mapping curve and the logarithmic domain mapping curve of the image block. By using the fusion curve method, it is helpful to control the contrast enhancement amplitude of different regions, further achieving the goal of improving image quality.

[0010] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0011] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0012] Figure 1 A schematic diagram of the image processing stage according to an embodiment of the present disclosure is shown;

[0013] Figure 2 A flowchart illustrating an image processing method according to an exemplary embodiment of the present disclosure is shown schematically;

[0014] Figure 3 A schematic diagram of dividing an image into blocks is shown;

[0015] Figure 4 It shows the relationship with Figure 3 A schematic diagram of the grayscale histograms of each corresponding image block;

[0016] Figure 5 It shows the relationship with Figure 4 A schematic diagram of the histogram mapping curves for each corresponding image patch;

[0017] Figure 6 A schematic diagram illustrating the fusion of a fusion mapping curve and a linear curve to generate a target mapping curve according to an embodiment of the present disclosure is shown.

[0018] Figure 7 This diagram illustrates a bilinear interpolation of a pixel in an image to be processed, according to an embodiment of the present disclosure.

[0019] Figure 8 A schematic diagram of an interactive interface for adjusting image processing intensity according to an embodiment of the present disclosure is shown;

[0020] Figure 9 The flowchart illustrating the entire process of an image processing method according to an embodiment of the present disclosure is shown in the schematic diagram.

[0021] Figure 10 A block diagram of an image processing apparatus according to an exemplary embodiment of the present disclosure is shown schematically;

[0022] Figure 11 A block diagram of an electronic device according to an exemplary embodiment of the present disclosure is shown schematically. Detailed Implementation

[0023] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0024] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0025] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all steps. For example, some steps may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances. Furthermore, all terms such as "first," "second," and "third" used below are for distinction purposes only and should not be construed as limiting the scope of this disclosure.

[0026] While CLAHE (Contrast Limited Adaptive Histogram Equalization) significantly improves image contrast, it lacks precise control over different image regions. Different contrast enhancements for different regions are necessary. For example, brighter areas can benefit from greater contrast enhancement, while enhancing darker areas may amplify noise; therefore, darker areas are unsuitable for significant contrast enhancement. Furthermore, some interpolation algorithms apply the same processing to all image regions, resulting in poor local representation and overall image quality.

[0027] In view of this, the present disclosure provides a new image processing scheme to improve the local contrast of an image and enhance image quality.

[0028] The image processing scheme of this disclosure can be implemented by an electronic device. That is, the electronic device can execute each step of the image processing method described below, and the image processing apparatus described below can be configured within the electronic device. For example, the image processing scheme of this disclosure can be implemented by an image signal processor equipped in the electronic device. In addition, this disclosure does not limit the type of electronic device, and may include, but is not limited to, smartphones, tablets, smart wearable devices, personal computers, servers, etc.

[0029] Figure 1 A schematic diagram illustrating the stage at which the image processing scheme of this disclosure embodiment is applied is shown. (Reference) Figure 1 In this embodiment, the input image is the image to be processed, that is, the original image to be enhanced. The input image can be an image captured by a camera module equipped with an electronic device, an image acquired by the electronic device from an external source (i.e., another device), an image generated by the electronic device in response to a user's drawing operation, or an image generated by the electronic device itself in response to other triggering events, etc. This disclosure does not limit the image source, image content, image size, etc. of the input image.

[0030] The image processing procedure of this disclosure can be used to process the input image to obtain an enhanced image corresponding to the input image. Subsequently, the electronic device can store, display, or perform further processing on the enhanced image. Such further processing includes, for example, denoising, object recognition, object tracking, and beautification. This disclosure does not limit the subsequent applications of the enhanced image.

[0031] In addition, the input image can be preprocessed before being processed using the image processing procedure of this disclosure. This disclosure does not limit the type of preprocessing, but may include, for example, denoising and brightness enhancement.

[0032] The image processing method of exemplary embodiments of this disclosure will now be described with reference to the accompanying drawings.

[0033] Figure 2 A flowchart illustrating an exemplary embodiment of the image processing method of this disclosure is shown schematically. Reference Figure 2 The image processing method may include the following steps:

[0034] S20. Obtain the image to be processed and divide it into multiple image blocks.

[0035] In exemplary embodiments of this disclosure, the image to be processed may be an image captured by a camera module equipped with an electronic device, an image acquired by the electronic device from an external source, an image generated by the electronic device in response to a user's drawing operation, or an image drawn automatically by the electronic device in response to other triggering events, etc. This disclosure does not limit the image source, image content, image size, etc. of the image to be processed.

[0036] Alternatively, the image to be processed can be a region of the original image, such as a region of interest to the user or an image region containing a specified object (such as a face) as determined by the electronic device itself. In this case, foreground segmentation can be performed on the original image to obtain the image to be processed.

[0037] Electronic devices can divide an image to be processed into multiple image blocks. For example, they can divide it into multiple square or rectangular blocks of the same size, or they can divide the image to be processed into image blocks of irregular shapes. This disclosure does not limit this.

[0038] refer to Figure 3 Electronic devices can divide the image to be processed into 6×8 image blocks of the same size.

[0039] S22. Determine the grayscale histogram of each image block, and determine the histogram mapping curve of each image block based on the grayscale histogram of each image block.

[0040] In determining the gray-level histogram of an image patch, the image's gray levels can be divided into bins at equal intervals. num There are several levels. For example, for an image with pixel grayscale values ​​of 0-255, if bin... num If the step size is set to 64, then the step size for each grayscale level is step = 256 / bin. num That is, step = 4. In other words, grayscale values ​​0-3 are counted in the Histogram. luma [0], 4-7 were counted in Hist luma [1], ..., 252-255 were counted in Hist. luma [bin num -1].

[0041] According to some embodiments of this disclosure, the histogram directly calculated for an image block can be used as the grayscale histogram of that image block.

[0042] According to some other embodiments of this disclosure, the directly calculated histogram can also be optimized, and the optimized histogram can be determined as the grayscale histogram of the image block.

[0043] First, the electronic device can perform pixel grayscale statistics on the image block to obtain the original grayscale histogram of the image block.

[0044] Next, the electronic device can apply threshold constraints and / or smoothing to the original grayscale histogram to determine the grayscale histogram of the image patch.

[0045] For threshold constraints, upper and lower thresholds can be pre-configured so that the statistical data represented by the grayscale histogram is constrained between the upper and lower thresholds.

[0046] Smoothing can make histogram transitions more reasonable, avoiding image degradation caused by abnormalities in the image itself or by processing errors. For example, for adjacent gray levels a, b, and c in the histogram, if the statistical values ​​of a and c are large, while the statistical value of b is very small, the histogram curve will not be smooth. In this case, the differences in the values ​​of a, b, and c can be smoothed.

[0047] Figure 4 To and Figure 3 The corresponding grayscale histograms of each image block are obtained using the above histogram statistical method.

[0048] After determining the grayscale histogram of each image block, the electronic device can perform histogram equalization on the grayscale histogram of the image block to generate a histogram mapping curve for the image block.

[0049] Specifically, this can be achieved by calculating the cumulative distribution function, as shown in Formulas 1 and 2:

[0050]

[0051]

[0052] Where Max is the maximum grayscale value of a pixel in the image to be processed; s num The number of sampling points used to generate the mapping curve, for example, cdf is the generated histogram mapping curve; HistSum can be calculated using formula 3:

[0053]

[0054] Figure 5 It shows the relationship with Figure 4 A schematic diagram of the histogram mapping curves for each corresponding image patch.

[0055] S24. Determine the maximum and minimum gray values ​​of each image block, and use the maximum and minimum gray values ​​of each image block to determine the logarithmic domain mapping curve of each image block.

[0056] In an exemplary embodiment of this disclosure, the grayscale maximum and minimum values ​​of an image block include the maximum and minimum grayscale values ​​of the pixels in the image block.

[0057] Electronic devices can determine the maximum grayscale value of pixels in an image to be processed, and use the maximum grayscale value of pixels in the image to be processed, the maximum grayscale value of pixels in an image patch, and the minimum grayscale value of pixels in the image patch to determine the logarithmic domain mapping curve of the image patch. The logarithmic domain mapping curve can also be called the log domain mapping curve.

[0058] First, the electronic device can determine the number of sampling points used to generate the logarithmic domain mapping curve based on the maximum grayscale value of the pixels in the image to be processed and the sampling step size. The sampling step size can be the step size of the grayscale levels in the grayscale histogram, and the number of sampling points can be as described in step S22.

[0059] Next, the electronic device can determine multiple coordinate points using the maximum grayscale value of pixels in the image to be processed, and the maximum and minimum grayscale values ​​of pixels in the image block. It can be understood that the number of coordinate points is the same as the number of sampling points mentioned above, i.e., the number of coordinate points determined is the number of sampling points.

[0060] Then, the electronic device can use these coordinate points to determine the logarithmic domain mapping curve of the image patch.

[0061] For example, the logarithmic field mapping curve of this disclosure embodiment can be obtained using the following formula 4:

[0062]

[0063] Where f(v) is the logarithmic domain mapping curve; i and j are the coordinate indices of the image patch in the image to be processed; V max (i, j) represents the maximum gray value of a pixel in the (i, j)-th image block; V min (i, j) represents the minimum gray value of the pixel in the (i, j)th image block; β is a manually set and adjustable parameter that can be used to prevent the occurrence of log0 and adjust the shape of the curve.

[0064] According to some embodiments of this disclosure, in formula 4, v = 0, 1, 2, ... Max.

[0065] According to other embodiments of this disclosure, in the case of determining the logarithmic domain mapping curve using sampling points as described above, v = 0, 1*step, 2*step, ..., (s num -2)*step, Max.

[0066] This disclosure utilizes a scheme to determine the logarithmic domain mapping curve using sampling points, which does not affect the image processing results or has an impact on the image processing results within an acceptable range. Since this process does not require processing every single point, it can save hardware costs.

[0067] Furthermore, it should be noted that the execution order of steps S22 and S24 is not limited in this embodiment.

[0068] S26. Determine the fusion mapping curve for each image block based on the histogram mapping curve and the logarithmic domain mapping curve for each image block.

[0069] It is understood that gain is the ratio of output to input. In an exemplary embodiment of this disclosure, the gain of the curve is the ratio of the dependent variable to the independent variable.

[0070] For each image block, firstly, the electronic device can determine the gain of the histogram mapping curve obtained in step S22, which can be expressed as Equation 5:

[0071]

[0072] The electronic device can also determine the gain of the logarithmic domain mapping curve obtained in step S24, which can be expressed as Equation 6:

[0073]

[0074] Next, the electronic device can determine the fusion mapping curve of the image patch using the gain of the histogram mapping curve and the gain of the logarithmic domain mapping curve. The fusion mapping curve tmf(v) can be expressed as Equation 7:

[0075] tmf(v)=(1+w log *(gain log (v)-1))*(1+w cdf *(gain cdf (v)-1))*v (Formula 7)

[0076] Among them, w log with w cdf These are manually set and adjustable parameters. In the point-by-point calculation scheme, v = 0, 1, 2, ... Max; in the sampling point calculation scheme, v = 0, 1*step, 2*step, ..., (s) num -2)*step, Max.

[0077] S28. Use the fusion mapping curve of each image patch to perform image enhancement on the image to be processed.

[0078] It should be understood that the image enhancement process in this embodiment of the disclosure applies to each pixel in the image to be processed. That is, although the process is described below using a single pixel as an example, the following image enhancement process can be performed on each pixel in the image to be processed.

[0079] First, determine the original grayscale value of the pixel and its position in the image block.

[0080] In some embodiments of this disclosure, taking a square image block as an example, the side length of the image block is mapped to 1. Thus, the position of any pixel in the image block can be represented, and after determining the position of the pixel in its own image block, the position of the pixel relative to other image blocks can also be determined.

[0081] Additionally, the set of image blocks associated with the pixel can be determined, and the target mapping curve of the image blocks in the set of image blocks can be obtained.

[0082] An image block set can be a collection of image blocks associated with the location of a pixel. The image block set can include one or more image blocks. For example, it can include only the image block to which the pixel belongs, or it can include multiple image blocks that are adjacent to the pixel.

[0083] The target mapping curve of the image patch can be determined based on the fusion mapping curve of the image patch determined in step S26 above.

[0084] In some embodiments of this disclosure, the fusion mapping curve of the image patch can be directly determined as the target mapping curve of the image patch and applied to the image enhancement process in this step.

[0085] In other embodiments of this disclosure, the fusion mapping curve of the image patch can be fused with a linear curve, and the fused curve can be used as the target mapping curve of the image patch. The linear curve can be a straight line where the independent variable and the dependent variable are equal, i.e., the linear curve can be f(v) = v. Furthermore, the linear curve can also be other forms of curves; for example, Formula 8 also defines a linear curve applicable to embodiments of this disclosure:

[0086]

[0087] This disclosure does not restrict the type of linear curve.

[0088] For the fusion process of the fusion mapping curve and the linear curve, firstly, the electronic device can determine the gray-level statistics of the image block. These gray-level statistics include the gray-level mean and gray-level variance of the image block. The gray-level mean is the average gray-level value of the pixels in the image block, and the gray-level variance is the variance of the gray-level values ​​of the pixels in the image block.

[0089] Next, the electronic device can combine the grayscale statistics of the image block to determine the fusion weight used when fusing the fusion mapping curve of the image block with the linear curve.

[0090] On one hand, the mean gray level of the image patch can be used as the independent variable of the first function to determine its value. On the other hand, the variance of the gray level of the image patch can be used as the independent variable of the second function to determine its value. Subsequently, based on the function values ​​of the first and second functions, the fusion weights used when fusing the fusion mapping curve of the image patch with the linear curve are determined.

[0091] The first function and the second function can be functions with the same form but different parameters. Specifically, the first function and the second function can be configured as multi-segment functions.

[0092] For example, if we denote the first function as g1 and the second function as g2, then both g1 and g2 can be configured as the following piecewise linear mapping functions:

[0093]

[0094] Thr1, Thr0, V1, and V0 are all manually set parameters, and these parameters can be different in g1 and g2.

[0095] The fusion weights used when fusing the fusion mapping curve of this image patch with the linear curve can be expressed as Equation 10:

[0096] w = g1(V mean (i,j))*g2(V var (i, j)) (Formula 10)

[0097] Among them, V mean (i, j) represents the average gray value of the (i, j)th image patch, V var (i, j) represents the grayscale variance of the (i, j)th image patch.

[0098] After determining the fusion weights, the target mapping curve can be expressed as Equation 11:

[0099] tmf final [v]=w*tmf(v)+(1-w)*f(v) (Formula 11)

[0100] Figure 6 A schematic diagram illustrating the fusion of a fusion mapping curve and a linear curve to generate a target mapping curve is shown, according to an embodiment of the present disclosure.

[0101] By combining fusion weights to fuse the fusion mapping curve with the linear curve, and by determining the fusion weights based on the statistical values ​​of the image blocks, electronic devices can adaptively adjust the image performance according to different image blocks.

[0102] After determining the original grayscale value of a pixel, its position in the image block it belongs to, and the target mapping curve of each image block in the image block set, the processed grayscale value of the pixel can be determined based on these data.

[0103] First, the electronic device can determine the distance weight of an image block in the image block set relative to the pixel based on the pixel's position within its respective image block. Specifically, the distance weight can be determined based on the relative position of the pixel to the center point of the image block in the image block set.

[0104] Furthermore, electronic devices can use the original grayscale value of a pixel and the target mapping curve of an image patch in a set of image patches to determine the function value of the target mapping curve of the image patch corresponding to that pixel. In other words, the original grayscale value is substituted into the formula of the target mapping curve to calculate the corresponding function value.

[0105] Next, the electronic device can determine the processed grayscale value of the pixel by combining the distance weight of the image block in the image block set relative to the pixel and the function value of the target mapping curve of the image block in the image block set corresponding to the pixel.

[0106] In some embodiments of this disclosure, regarding weighting, the grayscale value of a pixel after processing can be determined solely based on the distance weight.

[0107] The following is combined with Figure 7 The process of processing pixel O is illustrated by example.

[0108] For pixel O, the associated set of image blocks is determined as image block A, image block B, image block C, and its associated image block D. Each image block corresponds to a target mapping curve, which may lead to block artifacts. Therefore, bilinear interpolation can be used to eliminate block artifacts.

[0109] Let the original gray value of pixel O be denoted as r0, and the target mapping curves of image blocks A, B, C, and D be denoted as g, respectively. A g B g C g D Then, the gray value of pixel O after processing, i.e., the result of bilinear interpolation, can be expressed as Equation 12:

[0110] s o = (1-x)*(1-y)*g A (r0)+x*(1-y)*g B (r0)+(1-x)*y*g C (r0)+x*y*g D (r0) (Formula 12)

[0111] In some other embodiments of this disclosure, in addition to distance weight, brightness similarity weight can also be used as one of the weights to determine the grayscale value of the pixel after processing.

[0112] First, the electronic device can determine the average grayscale value of each image patch in the image patch set, and determine the difference between the grayscale value of the pixel and the average grayscale value of the image patches in the image patch set. Next, the electronic device can use this difference to determine the brightness similarity weight of the image patches in the image patch set relative to the pixel. Then, the electronic device can determine the processed grayscale value of the pixel based on the distance weight of the image patch in the image patch set relative to the pixel, the brightness similarity weight, and the function value of the target mapping curve of the image patch in the image patch set corresponding to the pixel.

[0113] Still referencing Figure 7 An example combining distance weight and brightness similarity weight will be described.

[0114] On the one hand, the distance weights of image blocks A, B, C, and D relative to pixel O are expressed as formulas 13 to 16 respectively:

[0115] w DA =(1-x)*(1-y) (Formula 13)

[0116] w DB =x*(1-y) (Formula 14)

[0117] w DC =(1-x)*y (Formula 15)

[0118] w DD =x*y (Formula 16)

[0119] On the other hand, firstly, the difference between the gray value of pixel O and the mean gray value of image blocks A, B, C, and D is denoted as S. A S B S C S D See formulas 17 to 20:

[0120] S A =|r o -V mean (A)| (Formula 17)

[0121] S B =|r o -V mean (B)| (Formula 18)

[0122] S C =|r o -V mean (C)| (Formula 19)S D =|r o -V mean (D)| (Formula 20)

[0123] Among them, V mean (A), V mean (B) V mean (C), V mean (D) represents the average gray values ​​of image blocks A, B, C, and D, respectively.

[0124] Based on the above differences, the weights w corresponding to brightness similarity were determined respectively. SA w SB w SC w SD See formulas 21 to 24:

[0125]

[0126]

[0127]

[0128]

[0129] Therefore, the new grayscale value of a pixel can be determined by combining distance weight and brightness similarity weight, and the corresponding interpolation formula can be expressed as Formula 25:

[0130]

[0131] The above-mentioned contrast enhancement scheme involves some manually set and adjustable parameters. According to other embodiments of this disclosure, these parameters can be integrated to form a unified adjustment method and presented on the interface of the electronic device so that users can adjust them according to different scenarios to meet their personalized needs.

[0132] refer to Figure 8 On the human-computer interface of electronic devices, there are options to adjust the intensity of contrast enhancement, allowing users to select the level of contrast enhancement.

[0133] The following is for reference. Figure 9 The processing procedure of one embodiment of the image processing method described above will be explained.

[0134] In step S902, the electronic device acquires the image to be processed and divides the image into multiple image blocks.

[0135] In step S904, the electronic device determines the grayscale histogram of each image block.

[0136] In step S906, the electronic device performs histogram equalization processing on the grayscale histogram to determine the histogram mapping curve of each image block.

[0137] In step S908, the electronic device determines the maximum and minimum gray values, the mean gray value, and the variance gray value for each image block.

[0138] In step S910, the electronic device uses the grayscale maximum and minimum values ​​to determine the logarithmic domain mapping curve of each image block.

[0139] In step S912, the electronic device fuses the histogram with the logarithmic domain mapping curve to obtain a fused mapping curve.

[0140] In step S914, the electronic device combines the fusion mapping curve with the linear curve by combining the fusion weight determined based on the gray mean and gray variance to obtain the target mapping curve.

[0141] In step S916, the electronic device uses the target mapping curve of each image block to interpolate and map the gray values ​​of each pixel in the image to be processed.

[0142] In step S918, the electronic device acquires and outputs the processed image. The processed image can then be stored, displayed, or subjected to further processing.

[0143] Furthermore, for input images in RGB, YUV, or Bayer input formats, the input image can be converted to a grayscale image to perform the image processing procedure described above. Moreover, the image processing method of this disclosure is applicable to images with pixel depths of 8 bits, 10 bits, 12 bits, etc., and has universality.

[0144] Through the above processing, it can be understood that the image processing method of the present disclosure can adjust not only the local contrast of the image, but also the local dynamics. The present disclosure does not limit these application scenarios.

[0145] It should be noted that although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0146] Furthermore, this example embodiment also provides an image processing apparatus.

[0147] Figure 10 A block diagram of an image processing apparatus according to an exemplary embodiment of the present disclosure is shown schematically. Reference Figure 10 The image processing apparatus 10 according to an exemplary embodiment of the present disclosure may include an image segmentation module 101, a first curve determination module 103, a second curve determination module 105, a third curve determination module 107, and an image enhancement module 109.

[0148] Specifically, the image segmentation module 101 can be used to acquire the image to be processed and divide it into multiple image blocks; the first curve determination module 103 can be used to determine the gray-level histogram of each image block and determine the histogram mapping curve of each image block based on the gray-level histogram of each image block; the second curve determination module 105 can be used to determine the gray-level extreme values ​​of each image block and determine the logarithmic domain mapping curve of each image block using the gray-level extreme values ​​of each image block; the third curve determination module 107 can be used to determine the fusion mapping curve of each image block based on the histogram mapping curve and the logarithmic domain mapping curve of each image block; and the image enhancement module 109 can be used to perform image enhancement on the image to be processed using the fusion mapping curve of each image block.

[0149] According to an exemplary embodiment of the present disclosure, the first curve determination module 103 may be configured to perform histogram equalization processing on the grayscale histogram of the image block to generate a histogram mapping curve of the image block.

[0150] According to an exemplary embodiment of the present disclosure, the first curve determination module 103 may also be configured to perform: pixel grayscale statistics on the image block to obtain the original grayscale histogram of the image block; and apply threshold constraints and / or smoothing processing to the original grayscale histogram to determine the grayscale histogram of the image block.

[0151] According to an exemplary embodiment of the present disclosure, the second curve determination module 105 may be configured to perform: determining the maximum value of the grayscale value of a pixel in the image to be processed; and determining the logarithmic domain mapping curve of each image block using the maximum value of the grayscale value of the pixel in the image to be processed, the maximum value of the grayscale value of the pixel in the image block, and the minimum value of the grayscale value of the pixel in the image block.

[0152] According to an exemplary embodiment of the present disclosure, the second curve determination module 105 may also be configured to perform: determining the number of sampling points based on the maximum gray value of the pixels in the image to be processed and the sampling step size; determining multiple coordinate points using the maximum gray value of the pixels in the image to be processed, the maximum gray value of the pixels in the image block, and the minimum gray value of the pixels in the image block, wherein the number of coordinate points is the number of sampling points; and determining the logarithmic domain mapping curve of the image block using the multiple coordinate points.

[0153] According to an exemplary embodiment of the present disclosure, the third curve determination module 107 may be configured to perform: determining the gain of the histogram mapping curve of the image patch; determining the gain of the logarithmic domain mapping curve of the image patch; and determining the fusion mapping curve of the image patch using the gain of the histogram mapping curve of the image patch and the gain of the logarithmic domain mapping curve of the image patch.

[0154] According to an exemplary embodiment of the present disclosure, the image enhancement module 109 can be configured to perform the following: for each pixel in the image to be processed, determine the original grayscale value of the pixel and the position of the pixel in the image block to which it belongs; determine the set of image blocks associated with the pixel and obtain the target mapping curve of the image block in the set of image blocks, wherein the target mapping curve of the image block is determined based on the fusion mapping curve of the image block; and determine the grayscale value of the pixel after processing according to the original grayscale value of the pixel, the position of the pixel in the image block to which it belongs and the target mapping curve of the image block in the set of image blocks.

[0155] According to an exemplary embodiment of this disclosure, the image enhancement module 109 may also be configured to perform: determining the distance weight of the image block in the image block set relative to the pixel based on the position of the pixel in its respective image block; determining the function value of the target mapping curve of the image block in the image block set corresponding to the pixel using the original gray value of the pixel and the target mapping curve of the image block in the image block set; and determining the processed gray value of the pixel by combining the distance weight of the image block in the image block set relative to the pixel and the function value of the target mapping curve of the image block in the image block set corresponding to the pixel.

[0156] According to an exemplary embodiment of this disclosure, the image enhancement module 109 may also be configured to perform: determining the average gray value of image blocks in the image block set; determining the difference between the gray value of a pixel and the average gray value of image blocks in the image block set; using the difference to determine the brightness similarity weight of image blocks in the image block set relative to the pixel; and determining the processed gray value of the pixel based on the distance weight, brightness similarity weight, and function value of the target mapping curve of the image block in the image block set corresponding to the pixel.

[0157] According to an exemplary embodiment of the present disclosure, the image enhancement module 109 may also be configured to perform: fusing the fusion mapping curve of the image patch with a linear curve to obtain the target mapping curve of the image patch.

[0158] According to an exemplary embodiment of this disclosure, a linear curve is a straight line in which the independent variable is equal to the dependent variable.

[0159] According to an exemplary embodiment of the present disclosure, the image enhancement module 109 may also be configured to perform: determining the grayscale statistics of an image block; combining the grayscale statistics of the image block to determine the fusion weight used when fusing the fusion mapping curve of the image block with the linear curve; and using the fusion weight to fuse the fusion mapping curve of the image block with the linear curve to obtain the target mapping curve of the image block.

[0160] According to an exemplary embodiment of this disclosure, the statistical values ​​of the grayscale values ​​of pixels in an image block include the grayscale mean and grayscale variance of the image block.

[0161] According to an exemplary embodiment of this disclosure, the image enhancement module 109 may also be configured to perform: using the gray-level mean of the image patch as the independent variable of a first function, determining the function value of the first function; using the gray-level variance of the image patch as the independent variable of a second function, determining the function value of the second function; and determining the fusion weight used when fusing the fusion mapping curve of the image patch with the linear curve based on the function values ​​of the first function and the second function; wherein the first function and the second function are functions with the same form but different parameter values.

[0162] Since the functional modules of the image processing apparatus of this disclosure are the same as those in the above-described method embodiments, they will not be described again here.

[0163] Figure 11 A schematic diagram of an electronic device suitable for implementing exemplary embodiments of the present disclosure is shown. It should be noted that... Figure 11 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0164] The electronic device disclosed herein includes at least a processor and a memory, the memory being used to store one or more programs, which, when executed by the processor, enable the processor to implement the image processing method of the exemplary embodiments of this disclosure.

[0165] Specifically, such as Figure 11 As shown, the electronic device 110 may include: a processor 1110, internal memory 1121, external memory interface 1122, Universal Serial Bus (USB) interface 1130, charging management module 1140, power management module 1141, battery 1142, antenna 1, antenna 2, mobile communication module 1150, wireless communication module 1160, audio module 1170, sensor module 1180, display screen 1190, camera module 1191, indicator 1192, motor 1193, buttons 1194, and a Subscriber Identification Module (SIM) card interface 1195, etc. The sensor module 1180 may include a depth sensor, pressure sensor, gyroscope sensor, barometric pressure sensor, magnetic sensor, accelerometer, distance sensor, proximity sensor, fingerprint sensor, temperature sensor, touch sensor, ambient light sensor, and bone conduction sensor, etc.

[0166] It is understood that the structures illustrated in the embodiments of this disclosure do not constitute a specific limitation on the electronic device 110. In other embodiments of this disclosure, the electronic device 110 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0167] Processor 1110 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors. Additionally, processor 1110 may include memory for storing instructions and data.

[0168] The electronic device 110 can implement shooting functions through an ISP, camera module 1191, video codec, GPU, display screen 1190, and application processor. In some embodiments, the electronic device 110 may include one or N camera modules 1191, where N is a positive integer greater than 1. If the electronic device 110 includes N cameras, one of the N cameras is the main camera.

[0169] Internal memory 1121 can be used to store computer executable program code, which includes instructions. Internal memory 1121 may include a program storage area and a data storage area. External memory interface 1122 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of electronic device 110.

[0170] This disclosure also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device.

[0171] Computer-readable storage media can be, for example—but not limited to—electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0172] A computer-readable storage medium can be sent, propagated, or transmitted for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wireless, wireline, optical fiber, RF, etc., or any suitable combination thereof.

[0173] A computer-readable storage medium carries one or more programs that, when executed by an electronic device, cause the electronic device to perform the methods described in the embodiments of this disclosure.

[0174] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0175] The units described in the embodiments of this disclosure can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the unit itself.

[0176] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0177] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0178] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0179] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0180] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized by, The method comprises the following steps: acquiring a to-be-processed image, and dividing the to-be-processed image into a plurality of image blocks; determining a gray level histogram of each of the image blocks, and determining a histogram mapping curve of each of the image blocks according to the gray level histogram of each of the image blocks; determining a gray level maximum value of each of the image blocks, and determining a logarithmic domain mapping curve of each of the image blocks by using the gray level maximum value of each of the image blocks; determining a fusion mapping curve of each of the image blocks according to the histogram mapping curve and the logarithmic domain mapping curve of each of the image blocks, comprising: determining a gain of the histogram mapping curve of an image block; determining a gain of the logarithmic domain mapping curve of the image block; and determining the fusion mapping curve of the image block by using the gain of the histogram mapping curve of the image block and the gain of the logarithmic domain mapping curve of the image block; performing image enhancement on the to-be-processed image by using the fusion mapping curve of each of the image blocks, comprising: for each pixel point in the to-be-processed image, determining an original gray level value of the pixel point and a position of the pixel point in a corresponding image block; determining a set of image blocks associated with the pixel point, and acquiring a target mapping curve of an image block in the set of image blocks, wherein the target mapping curve of the image block is determined based on the fusion mapping curve of the image block; and determining a processed gray level value of the pixel point according to the original gray level value of the pixel point, the position of the pixel point in the corresponding image block, and the target mapping curve of the image block in the set of image blocks.

2. The image processing method of claim 1, wherein, The method comprises the following steps: performing histogram equalization processing on the gray level histogram of the image block to generate the histogram mapping curve of the image block.

3. The image processing method of claim 1 or 2, characterized in that, The method comprises the following steps: performing pixel gray level statistics on the image block to obtain an original gray level histogram of the image block; performing threshold constraint and / or smoothing processing on the original gray level histogram to determine the gray level histogram of the image block.

4. The image processing method of claim 1, wherein, The method comprises the following steps: determining a maximum value of the gray level values of the pixel points in the to-be-processed image; determining the logarithmic domain mapping curve of each of the image blocks by using the maximum value of the gray level values of the pixel points in the to-be-processed image, the maximum value and the minimum value of the gray level values of the pixel points in the image block.

5. The image processing method of claim 4, wherein, The method comprises the following steps: determining a number of sampling points according to the maximum value of the gray level values of the pixel points in the to-be-processed image and a sampling step length; determining a plurality of coordinate points by using the maximum value of the gray level values of the pixel points in the to-be-processed image, the maximum value and the minimum value of the gray level values of the pixel points in the image block, wherein the number of the coordinate points is equal to the number of the sampling points; determining the logarithmic domain mapping curve of the image block by using the plurality of coordinate points.

6. The image processing method of claim 1, wherein, According to the original gray value of the pixel point, the position of the pixel point in the image block to which the pixel point belongs, and the target mapping curve of the image blocks in the image block set, the processed gray value of the pixel point is determined, comprising: According to the position of the pixel point in the image block to which the pixel point belongs, the distance weight of the image blocks in the image block set relative to the pixel point is determined; Using the original gray value of the pixel point and the target mapping curve of the image blocks in the image block set, the function value of the target mapping curve of the image blocks in the image block set corresponding to the pixel point is determined; The distance weight of the image blocks in the image block set relative to the pixel point and the function value of the target mapping curve of the image blocks in the image block set corresponding to the pixel point are combined to determine the processed gray value of the pixel point.

7. The image processing method of claim 6, wherein, The distance weight of the image blocks in the image block set relative to the pixel point and the function value of the target mapping curve of the image blocks in the image block set corresponding to the pixel point are combined to determine the processed gray value of the pixel point, comprising: The gray mean value of the image blocks in the image block set is determined; The difference between the gray value of the pixel point and the gray mean value of the image blocks in the image block set is determined; Using the difference, the brightness similarity weight of the image blocks in the image block set relative to the pixel point is determined; According to the distance weight of the image blocks in the image block set relative to the pixel point, the brightness similarity weight, and the function value of the target mapping curve of the image blocks in the image block set corresponding to the pixel point, the processed gray value of the pixel point is determined.

8. The image processing method of claim 1, wherein, The image processing method further comprises: The fusion mapping curve of the image block is fused with the linear curve to obtain the target mapping curve of the image block.

9. The image processing method of claim 8, wherein, The linear curve is a straight line where the independent variable is equal to the dependent variable.

10. The image processing method of claim 8, wherein, The image processing method further comprises: The gray statistical value of the image block is determined; The fusion weight used when the fusion mapping curve of the image block is fused with the linear curve is determined in combination with the gray statistical value of the image block; The fusion mapping curve of the image block is fused with the linear curve using the fusion weight to obtain the target mapping curve of the image block.

11. The image processing method of claim 10, wherein, The statistical value of the gray value of the pixel points in the image block includes the gray mean value and the gray variance of the image block.

12. The image processing method of claim 11, wherein, The fusion weight used when the fusion mapping curve of the image block is fused with the linear curve is determined in combination with the gray statistical value of the image block, comprising: The gray mean value of the image block is taken as the independent variable of the first function to determine the function value of the first function; The gray variance of the image block is taken as the independent variable of the second function to determine the function value of the second function; According to the function value of the first function and the function value of the second function, the fusion weight used when the fusion mapping curve of the image block is fused with the linear curve is determined; Wherein, the first function and the second function are functions with the same form but different parameter values.

13. An image processing apparatus characterized by comprising: Comprising: An image division module is configured to obtain a to-be-processed image and divide the to-be-processed image into a plurality of image blocks. a first curve determining module configured to determine a gray scale histogram of each of the image blocks, and determine a histogram mapping curve of each of the image blocks according to the gray scale histogram of each of the image blocks; a second curve determining module configured to determine a gray scale extreme value of each of the image blocks, and determine a log domain mapping curve of each of the image blocks by using the gray scale extreme value of each of the image blocks; a third curve determining module configured to determine a fusion mapping curve of each of the image blocks according to the histogram mapping curve and the log domain mapping curve of each of the image blocks, including: determining a gain of the histogram mapping curve of an image block; determining a gain of the log domain mapping curve of the image block; and determining the fusion mapping curve of the image block by using the gain of the histogram mapping curve of the image block and the gain of the log domain mapping curve of the image block; an image enhancement module configured to perform image enhancement on the image to be processed by using the fusion mapping curve of each of the image blocks, including: determining an original gray scale value of each pixel in the image to be processed and a position of the pixel in a corresponding image block; determining a set of image blocks associated with the pixel, and obtaining a target mapping curve of an image block in the set of image blocks, where the target mapping curve of the image block is determined based on the fusion mapping curve of the image block; and determining a processed gray scale value of the pixel according to the original gray scale value of the pixel, the position of the pixel in the corresponding image block, and the target mapping curve of the image block in the set of image blocks.

14. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program, when executed by a processor, implements the image processing method of any one of claims 1-12.

15. An electronic device, comprising: comprising: a processor; a memory configured to store one or more programs, which, when executed by the processor, cause the processor to implement the image processing method of any one of claims 1-12.

Images