Noise estimation method, image processing method and device, and storage medium

Abstract

A noise estimation method, an image processing method and device, and a storage medium. In the noise estimation method, a mapping derivative result is obtained according to the mapping curve; an original estimation result is obtained, and the original estimation result is the noise estimation result of the original image; and a noise estimation result of the image to be estimated is obtained according to the mapping derivative result and the original estimation result. Performing noise estimation on the image to be estimated obtained after nonlinear mapping can effectively save hardware resources, and the noise estimation result is obtained according to the mapping derivative result related to the mapping curve of the nonlinear mapping and the original estimation result, so the noise estimation result is related to the nonlinear mapping, and the noise estimation result has higher accuracy, which can effectively improve the effect of subsequent noise reduction processing, and only needs to be derived for the mapping curve of the nonlinear mapping, so the calculation amount is small and the calculation speed is fast.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a noise estimation method, an image processing method and device, and a storage medium. Background Art

[0002] Accurate noise estimation of images is the basis for many functional modules in image processing to function properly. For example, the effects of 2D noise reduction and 3D noise reduction all depend on accurate noise estimation results.

[0003] Most commonly used image sensors are linear, that is, the digital signal increases linearly with the increase in the number of received photons. Current technology can provide relatively accurate noise estimation for the unprocessed original images (RAW images) obtained by linear image sensors, such as noise estimation using Poisson noise model and Gaussian noise model. This noise estimation method is based on the characteristic parameters of the image sensor, rather than just on the image itself, so the estimation result obtained by this method is very accurate.

[0004] This estimation method refers to the average brightness μ of all pixels in a small window in the obtained image, based on the parameters related to the characteristics and gain of the image sensor. x , estimate the noise variance var(x) of the pixel at the center of the small window. Specifically, the commonly used formula for noise estimation based on the Poisson noise model and the Gaussian noise model is:

[0005] var(x)=k ne ×μ x +b ne

[0006] Among them, k ne 、b ne are parameters related to the characteristics of the image sensor and gain, respectively, and can be determined experimentally. It can be seen that the noise estimation results based on the Poisson noise model and the Gaussian noise model are the average brightness μ x The function can be plotted as a curve in a coordinate system with average brightness as the horizontal axis and noise variance as the vertical axis.

[0007] The obtained noise variance can be used for subsequent image processing of the original image, such as for noise reduction processing to improve image quality. However, with the increasing popularity of high-dynamic image sensors, the data bit width of the captured original image is usually large, which can reach 16 bits or even 20 bits, resulting in a large amount of calculation, slow calculation speed, and high power consumption for noise reduction and other processing on the original image, which makes it difficult to use in mobile devices such as cameras and mobile phones, as well as applications with high real-time requirements. After the original image is subjected to nonlinear mapping such as tone mapping to compress the data bit width, although the data bit width is reduced, the mapping is nonlinear, resulting in a significant change in the noise morphology after mapping, which no longer conforms to the Poisson noise model and the Gaussian noise model. Therefore, the noise variance obtained based on the above method is no longer accurate, and using it for noise reduction processing is prone to problems such as poor noise reduction effect. Summary of the invention

[0008] The problem solved by the present invention is how to reduce the amount of calculation and increase the calculation speed while improving the accuracy of the noise estimation result and the noise reduction effect.

[0009] To solve the above problem, the present invention provides a noise estimation method, comprising:

[0010] An image to be estimated is obtained, wherein the image to be estimated is an image obtained after nonlinear mapping is performed on an original image, wherein the nonlinear mapping is a mapping of pixel values; a mapping curve of the nonlinear mapping is obtained, wherein the mapping curve is a curve that is at least first-order differentiable; according to the mapping curve, a mapping derivative result is obtained; an original estimation result is obtained, wherein the original estimation result is a noise estimation result of the original image; and a noise estimation result of the image to be estimated is obtained according to the mapping derivative result and the original estimation result.

[0011] Correspondingly, the present invention also provides a noise estimation device, comprising: a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the processor executes the steps of the noise estimation method of the present invention.

[0012] and a storage medium, wherein the storage medium is a non-volatile storage medium or a non-transient storage medium, on which computer instructions are stored, and when the computer instructions are executed, the steps of the noise estimation method of the present invention are executed.

[0013] In addition, the present invention also provides an image processing method, comprising: performing noise estimation and obtaining a noise estimation result, wherein in the step of performing noise estimation, noise estimation is performed by the noise estimation method of the present invention.

[0014] Optionally, the method further includes: performing noise reduction processing according to the noise estimation result.

[0015] Correspondingly, the present invention also provides an image processing device, comprising: a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the processor executes the steps of the image processing method of the present invention.

[0016] and a storage medium, wherein the storage medium is a non-volatile storage medium or a non-transient storage medium, on which computer instructions are stored, and when the computer instructions are executed, the steps of the image processing method of the present invention are executed.

[0017] Compared with the prior art, the technical solution of the present invention has the following advantages:

[0018] In the technical solution of the present invention, a mapping derivative result is obtained according to the mapping curve; an original estimation result is obtained, and the original estimation result is a noise estimation result of the original image; and a noise estimation result of the image to be estimated is obtained according to the mapping derivative result and the original estimation result. Performing noise estimation on the image to be estimated obtained after nonlinear mapping can effectively save hardware resources, and the noise estimation result is obtained based on the mapping derivative result related to the mapping curve of the nonlinear mapping and the original estimation result, so the noise estimation result is related to the nonlinear mapping, and the noise estimation result is more accurate, which can effectively improve the effect of subsequent noise reduction processing, and only needs to be derived for the mapping curve of the nonlinear mapping, so the calculation amount is small and the calculation speed is fast.

[0019] In an optional solution of the present invention, an inverse mapping curve is obtained according to the mapping curve of the nonlinear mapping; and the original estimation result is obtained according to the inverse mapping curve in combination with the image to be estimated. Based on the inverse mapping curve of the mapping curve and in combination with the image to be estimated, the original estimation result of the original image is inferred, so that the original image does not need to be stored, a large amount of storage data, delay and reading process can be saved, and hardware resources such as storage, bandwidth and power consumption can be effectively saved, which is more conducive to implementation on hardware platforms such as FPGA. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a schematic flow chart of a first embodiment of a noise estimation method of the present invention;

[0021] Figure 2 yes Figure 1 The image to be estimated in the first embodiment of the noise estimation method of the present invention is shown;

[0022] Figure 3 yes Figure 1 The original image in the first embodiment of the noise estimation method of the present invention is shown;

[0023] Figure 4 yes Figure 1 A family of mapping curves of tone mapping described in the first embodiment of the noise estimation method of the present invention is shown;

[0024] Figure 5 yes Figure 1 The flowchart of the step of obtaining the original estimation result in the first embodiment of the noise estimation method of the present invention is shown;

[0025] Figure 6 yes Figure 1 The original estimation result obtained in the first embodiment of the noise estimation method of the present invention is shown;

[0026] Figure 7 yes Figure 1 The noise estimation result of the image to be estimated obtained in the first embodiment of the noise estimation method of the present invention is shown;

[0027] Figure 8 is a flow chart of the steps of obtaining the original estimation result in the second embodiment of the noise estimation method of the present invention;

[0028] Fig. 9 yes Figure 8 The flowchart of the step of obtaining the original estimation result in the second embodiment of the noise estimation method of the present invention is shown;

[0029] Fig.10 yes Figure 8 The noise estimation result obtained in the second embodiment of the noise estimation method of the present invention is shown;

[0030] Fig.11 is the noise estimation result obtained in the third embodiment of the noise estimation method of the present invention;

[0031] Fig.12 is the noise estimation result obtained in the fourth embodiment of the noise estimation method of the present invention;

[0032] Fig.13 is the noise estimation result obtained in the fifth embodiment of the noise estimation method of the present invention;

[0033] Fig.14 is the noise estimation result obtained in the sixth embodiment of the noise estimation method of the present invention;

[0034] Fig.15 is the noise estimation result obtained in the seventh embodiment of the noise estimation method of the present invention;

[0035] Fig.16 is the noise estimation result obtained in the eighth embodiment of the noise estimation method of the present invention;

[0036] Fig.17is a mapping curve of the tone mapping described in the ninth embodiment of the noise estimation method of the present invention;

[0037] Fig.18 yes Fig.17 The noise estimation result obtained in the ninth embodiment of the noise estimation method of the present invention is shown;

[0038] Fig.19 is the noise estimation result obtained in the tenth embodiment of the noise estimation method of the present invention;

[0039] Fig. 20 is the image to be estimated obtained in the eleventh embodiment of the noise estimation method of the present invention;

[0040] Fig.21 yes Fig. 20 The original image in the eleventh embodiment of the noise estimation method of the present invention is shown;

[0041] Fig. 22 yes Fig. 20 The mapping curve of tone mapping in the eleventh embodiment of the noise estimation method of the present invention is shown;

[0042] Fig.23 yes Fig. 20 In the eleventh embodiment of the noise estimation method of the present invention, Fig. 22 The mapping derivative results obtained by the mapping curve shown;

[0043] Fig.24 yes Fig. 20 The noise estimation result of the original image in the eleventh embodiment of the noise estimation method of the present invention is shown;

[0044] Fig.25 yes Fig. 20 The noise estimation result obtained in the eleventh embodiment of the noise estimation method of the present invention is shown. DETAILED DESCRIPTION

[0045] As can be seen from the background technology, the noise reduction processing based on the noise variance obtained by the existing noise estimation technology has the problem of inaccurate noise estimation and poor noise reduction effect. Now, combined with an existing noise estimation method, the reasons for the inaccurate noise estimation and poor noise reduction effect are analyzed:

[0046] In order to save hardware resources, in existing image processing methods, the noise reduction processing step (such as 2D noise reduction processing, 3D noise reduction processing, etc.) is often performed after nonlinear mapping of the RAW image, for example, noise reduction processing is performed after tone mapping, where tone mapping is a nonlinear mapping of the pixel values ​​of the image.

[0047] After tone mapping, the noise form of the RAW image has changed greatly and no longer conforms to the Poisson noise model and the Gaussian noise model. If the noise estimation method of the RAW image is still used to perform noise estimation, the noise estimation result will no longer be accurate. If this inaccurate noise estimation result is used as an input parameter for 2D noise reduction or 3D noise reduction, it is easy to have poor noise reduction effects, such as image blur or failure to reduce noise.

[0048] It can be seen that how to accurately estimate the noise of the image after nonlinear mapping such as tone mapping is crucial to improving the noise reduction effect.

[0049] In the image, the pixel value of each pixel is discrete (in DN). According to the theory of random variables and the distribution law of a discrete random variable P(X=xi)=pi, the expectation and variance of its function can be obtained. In this way, the noise estimation result after nonlinear mapping can be obtained based on the noise estimation result before nonlinear mapping.

[0050] Specifically, in the random variable theory method, the process of obtaining the noise estimation result after nonlinear mapping is as follows: suppose the pixel value of any pixel position of the RAW image is a random variable X, the mapping curve of the nonlinear mapping is the function y=g(x), and the pixel value of the same pixel position of the image after nonlinear mapping is a random variable Y, then Y is a function of X: Y=g(X).

[0051] Theorem: Let X be a discrete random variable, whose distribution law is P(X=xi)=pi. If the series Absolute convergence, then the mathematical expectation of the univariate function Y = g(X) of X is

[0052] According to the above theorem, it can be obtained that the expectation of the pixel value of the pixel bit after nonlinear mapping is:

[0053]

[0054] as well as,

[0055]

[0056] Among them, y i =g(x i ),x i =0,1,2,…,x max , is the function represented by the nonlinear mapping, that is, the mapping relationship between the pixel values ​​of the corresponding pixel bits of the image before and after the nonlinear mapping, which is expressed as a mapping curve of the pixel values.

[0057] A well-designed mapping curve is usually strictly monotonically increasing, xmax is the maximum value of the pixel value range in the RAW image before nonlinear mapping. For example, when the RAW image is 8-bit data, x max =255; when the RAW image is 14-bit data, x max =2 14 -1; Due to the existence of noise, the pixel value X of any pixel before nonlinear mapping may be equal to 0, 1, 2, ..., x max This can be understood as a camera continuously taking pictures of a completely still scene with constant lighting using exactly the same parameters, but the pixel values ​​of the same pixel in each picture are different, randomly distributed according to a certain probability, p i It is the noise distribution law of the RAW image before nonlinear mapping, that is, the pixel value X is equal to a certain value x i Since the RAW image noise can usually be modeled as a Poisson noise model, a Gaussian noise model or a mixture of the two, the noise distribution law P(X=xi)=pi of the RAW image can be obtained according to the noise model, parameters related to the image sensor characteristics and the pixel value of the RAW image.

[0058] Based on E(Y) and E(Y 2 ), we can get the average pixel value of the image after nonlinear mapping: μ Y =E(Y); we can also get the noise variance: σ Y 2 =D(Y)=E(Y 2 )-[E(Y)] 2 . It can be obtained that the noise variance σ of the image after nonlinear mapping Y 2 and the average pixel value μ Y The corresponding relationship: σ Y 2 ~μ Y .

[0059] Based on the above correspondence, the noise estimation result can be obtained according to the average pixel value of each pixel of the image after nonlinear mapping, for example, the noise estimation result of the central pixel of the small window can be obtained according to the average pixel value of each pixel in the small window of 3×3 or 5×5. Among them, the method of obtaining the noise estimation result of the central pixel of the small window according to the average pixel value of multiple pixels in the small window can eliminate the influence of noise fluctuation on the pixel value of a single pixel.

[0060] However, the above method has a large amount of calculation and is difficult to use in situations where real-time output is required, such as cameras and mobile phones. For example, before nonlinear mapping, the RAW image noise satisfies the mean μ and variance σ. 2Normal distribution: X~N(μ,σ 2 ),but So calculate When , not only is the summation (Σ) computationally intensive, but also i It is related to the pixel value μ of each pixel bit of the RAW image. It is necessary to calculate each pixel value μ∈[0, x max ] are calculated once to obtain the noise variance corresponding to each pixel value of the image. Such a huge amount of calculation is difficult to implement in real-time applications.

[0061] In order to solve the technical problem, the present invention provides a noise estimation method, an image processing method and device, and a storage medium, which perform noise estimation on the image to be estimated obtained after nonlinear mapping, which can effectively save hardware resources, and the noise estimation result is obtained based on a mapping derivative result related to a mapping curve of the nonlinear mapping and the original estimation result, so the noise estimation result is related to the nonlinear mapping, the noise estimation result has higher accuracy, can effectively improve the effect of subsequent noise reduction processing, and only needs to be derived for the mapping curve of the nonlinear mapping, so the calculation amount is small and the calculation speed is fast.

[0062] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0063] refer to Figure 1 , shows a flow chart of an embodiment of a noise estimation method of the present invention.

[0064] The noise estimation method comprises: executing step S100 to obtain an image to be estimated, wherein the image to be estimated is an image obtained after nonlinear mapping is performed on an original image, wherein the nonlinear mapping is a mapping for pixel values, and the mapping curve is a curve that is at least first-order differentiable; executing step S200 to obtain a mapping curve of the nonlinear mapping; executing step S300 to obtain a mapping derivative result according to the mapping curve; executing step S400 to obtain an original estimation result, wherein the original estimation result is a noise estimation result of the original image; and executing step S500 to obtain a noise estimation result of the image to be estimated according to the mapping derivative result and the original estimation result.

[0065] Noise estimation is performed on the image to be estimated obtained after nonlinear mapping, so as to perform subsequent noise reduction processing, which can effectively save hardware resources. Moreover, the noise estimation result is related to the nonlinear mapping, and the noise estimation result has higher accuracy, which can effectively improve the effect of subsequent noise reduction processing. In addition, it is only necessary to derive the mapping curve of the nonlinear mapping, which has a small amount of calculation and a fast calculation speed.

[0066] The specific technical solution of the noise estimation method embodiment of the present invention is described in detail below with reference to the accompanying drawings.

[0067] refer to Figure 1 , combined with reference Figure 2 and Figure 3 First, step S100 is performed to obtain the image to be estimated 101 (such as Figure 2 As shown), the image to be estimated 101 is the original image 102 (as shown Figure 3 An image obtained after nonlinear mapping of the image (as shown), wherein the nonlinear mapping is a mapping of pixel values.

[0068] It should be noted that Figure 1 In the illustrated embodiment, tone mapping is used as an example for description. However, in other embodiments of the present invention, the nonlinear mapping may also be one of gamma correction, dynamic range compression and contrast adjustment.

[0069] like Figure 2 As shown, the image to be estimated 101 is an image obtained by tone mapping the original image 102 in a rational function transformation manner. Figure 3 As shown, the original image 102 is a grayscale image, so as to facilitate the statistical analysis of the noise variance of the grayscale regions of each brightness of the image. Specifically, the original image 102 is a grayscale raw image containing Poisson noise and Gaussian noise, with brightness ranging from 0 to 2 14 -1 is divided into 200 gray levels.

[0070] Continue to refer Figure 1 After obtaining the image 101 to be estimated, step S200 is performed to obtain a mapping curve of the nonlinear mapping, where the mapping curve is a curve that is at least first-order differentiable.

[0071] The nonlinear mapping is a mapping for image pixel values ​​(i.e., grayscale values, brightness values). The mapping curve refers to a pixel value mapping relationship curve of the image nonlinear transformation, i.e., a relationship curve between the image pixel values ​​before and after the mapping in the image nonlinear transformation. Therefore, the mapping curve of the nonlinear mapping can be expressed as:

[0072] y=g(x)

[0073] Here, x represents the pixel value of a pixel in the original image 102 , and y represents the pixel value of a pixel in the image to be estimated 101 .

[0074] Specific as Figure 2 and Figure 3In the illustrated embodiment, the nonlinear mapping is tone mapping, and the tone mapping is implemented in a rational function transformation manner. Therefore, in the step of performing step S200 to obtain a mapping curve of the nonlinear mapping, the mapping curve of the tone mapping is:

[0075]

[0076] Where x represents the pixel value of the original image pixel bit, x max represents the maximum pixel value allowed by the bit width of the original image, y represents the pixel value of the pixel bit of the image to be estimated, and y max represents the maximum pixel value allowed by the bit width of the image to be estimated, and a and b are adjustment parameters. By adjusting the values ​​of a and b, a family of mapping curves can be generated, such as Figure 4 As shown. The tone mapping is 14bit to 10bit, that is, x max =2 14 -1,y max =2 10 -1, b=2 12 , a as shown in the figure. Figure 2 and Figure 3 In the embodiment shown, a=2 9 .

[0077] In some embodiments of the present invention, in the step of executing step S200 to obtain the mapping curve of the nonlinear mapping, the mapping curve is monotonic, that is, when the pixel value of the pixel in the original image 102 increases or decreases, the pixel value of the pixel in the image to be estimated 101 increases or decreases accordingly. Figure 2 and Figure 3 In the illustrated embodiment, when the tone mapping is implemented in a rational function transformation manner, step S200 is performed to obtain a mapping curve of the tone mapping, and the mapping curve is monotonically increasing.

[0078] The mapping curve is at least a first-order differentiable curve, that is, the mapping curve y=g(x) has at least a first-order derivative. Therefore, after obtaining the mapping curve of the nonlinear mapping, the mapping curve y=g(x) is subjected to a first-order Taylor expansion near x=μ to obtain:

[0079] y=g(x)≈g(μ)+g′(μ)·(x-μ)

[0080] Among them, g'(μ) represents the first-order derivative of the mapping curve g(x) at x=μ. So we have:

[0081] E(y)≈g(μ)+g'(μ)·[E(x)-μ]=g(μ)

[0082] var(y)≈[g'(μ)]2 var(x)

[0083] Wherein, x represents the pixel value of any pixel position (i, j) in the original image 102, y represents the pixel value of the corresponding pixel position (i, j) in the image to be estimated 101, x and y are both random variables, var(x) represents the noise variance of the pixel value of any pixel position (i, j) in the original image 102, var(y) represents the noise variance of the pixel value of the corresponding pixel position (i, j) in the image to be estimated 101, E(y) represents the expectation of the pixel value of the image to be estimated 101 obtained after nonlinear mapping, and E(x) represents the expectation of the pixel value of the original image 102.

[0084] Therefore, the relationship between the noise variance of the image to be estimated 101 and the noise variance of the original image 102 can also be expressed as:

[0085]

[0086] Among them, σ x 2 (i, j) represents the noise variance of the pixel value of any pixel position (i, j) in the original image 102; σ y 2 (i, j) represents the noise variance of the pixel value of the corresponding pixel position (i, j) in the image 101 to be estimated.

[0087] It should be noted that, in theory, μ represents the mathematical expectation of the pixel value of any pixel position (i, j) in the original image 102: μ = E(x). However, in the actual image processing process, the exact mathematical expectation E(x) cannot be obtained. Therefore, in some embodiments of the present invention, μ represents the average value of the pixel values ​​in a small area around the pixel position (i, j) in the original image 102. For example, the small area is a 3×3, 5×5, or 7×7 small area centered on the pixel position (i, j). As an approximation, that is, μ represents the average value of the pixel values ​​in a small area centered on the pixel position (i, j) in the original image 102.

[0088] It can be seen that the noise variance var(y) of the image to be estimated 101 obtained after the nonlinear mapping is related to the derivative result of the mapping curve g(x) of the nonlinear mapping, so Figure 1 As shown, after the mapping curve is obtained, step S300 is performed to obtain a mapping derivative result according to the mapping curve.

[0089] In some embodiments of the present invention, in the step of obtaining the mapping derivative result, S300, the first-order derivative of the mapping curve of the nonlinear mapping is performed. Figure 2 and Figure 3 In the illustrated embodiment, the nonlinear mapping is tone mapping, and the tone mapping is implemented in a rational function transformation manner. According to the mapping curve of the rational function transformation, it can be known that in the step of performing step S300 to obtain the mapping derivative result, the mapping derivative result can be expressed as:

[0090]

[0091] On the other hand, according to the relationship between the noise variance of the image to be estimated 101 and the noise variance of the original image 102, it can be known that, in addition to being related to the derivative result of the mapping curve g(x), the noise variance var(y) of the image to be estimated 101 is also related to the noise of the original image 102, so Figure 1 As shown, the noise estimation method further includes: executing step S400 to obtain an original estimation result, where the original estimation result is a noise estimation result of the original image.

[0092] like Figure 5 As shown, in some embodiments of the present invention, executing step S400, the step of obtaining the original estimation result includes: executing step S411 to obtain the original image 102; executing step S412 to obtain the original estimation result based on the original image 102.

[0093] Combined with reference Figure 6 , showing Figure 1 The original estimation result obtained in the embodiment of the noise estimation method is shown.

[0094] Since the original image 102 is a grayscale image, Figure 6 As shown, in the step of executing step S412 to obtain the original estimation result, the original estimation result is obtained by counting the variance of the pixel values ​​within each grayscale range in the original image 102.

[0095] Specific as Figure 6 In the illustrated embodiment, the horizontal axis represents the average brightness of the original image 102, in units of DN; the vertical axis represents the noise variance of the original image 102, in units of DN 2 As shown in the figure, the noise estimation result of the raw image is a typical photon transfer curve of a RAW image.

[0096] Continue to refer Figure 1 After obtaining the mapping derivative result and the original estimation result, step S500 is performed to obtain the noise estimation result of the image to be estimated according to the mapping derivative result and the original estimation result.

[0097] Specifically, based on the relationship between the noise variance of the image to be estimated 101 and the noise variance of the original image 102, in some embodiments of the present invention, step S500 is executed, and in the step of obtaining the noise estimation result of the image to be estimated based on the mapping derivative result and the original estimation result, the noise estimation result of the image to be estimated is obtained based on the product of the square of the mapping derivative result and the original estimation result.

[0098] Specifically, in the step of executing step S500, in which the noise estimation result of the image to be estimated is obtained according to the mapping derivative result and the original estimation result, the noise variance of the image to be estimated is:

[0099]

[0100] in, represents the noise variance of any pixel (i, j) of the image to be estimated, σ x 2 (i, j) represents the noise variance of the same pixel position (i, j) of the original image, g'(μ) represents the mapping derivative result, and μ represents the average pixel value in a small area near the pixel position (i, j) in the original image.

[0101] Figure 7 Shows Figure 1 The noise estimation result of the image 101 to be estimated obtained in the embodiment of the noise estimation method shown in FIG. 1 is as follows; wherein: Figure 7 Also shown is a noise variance curve obtained by actual statistics of the image to be estimated 101.

[0102] Specifically, Figure 7 As shown, the horizontal axis represents the average pixel value of the image to be estimated 101, unit: DN; the vertical axis represents the noise variance of the image to be estimated 101, unit: DN 2 ; Circle data points represent Figure 1 The noise estimation result obtained by the embodiment of the noise estimation method is shown; the solid line represents the noise result obtained according to the actual statistics of the image 101 to be estimated.

[0103] Depend on Figure 7 You can know, Figure 1 The noise estimation result obtained by the embodiment of the noise estimation method shown is compared with the noise result actually obtained by statistics in the image to be estimated 101, and the two are almost completely overlapped, and the two are highly consistent, that is, the noise variance accuracy of the image to be estimated obtained by the embodiment of the noise estimation method is high, that is, the noise estimation method of the present invention can obtain a high-precision noise estimation result.

[0104] The reason why grayscale images are used in the above embodiments is that the pixel values ​​within any grayscale range are equivalent to the noise superimposed on the average pixel value of this grayscale. Therefore, by counting the average brightness and variance of the pixel values ​​within this grayscale range, the actual noise statistics corresponding to the average brightness can be obtained. Repeating this process for each grayscale, the actual noise statistics corresponding to each average brightness can be obtained, so that the actual noise statistics can be used to compare with the noise estimation results obtained by the noise estimation method of the present invention. In actual work, the noise estimation method of the present invention is not only applicable to grayscale images, but also to images of any scene.

[0105] It should be noted that Figures 1 to 7 In the illustrated embodiment, the original image 102 is a grayscale image, and therefore, in the step of obtaining the original estimation result, the original estimation result is obtained according to the variance of the pixel values ​​within each grayscale range in the original image 102. However, this method of obtaining the original estimation result is only an example, and in other embodiments of the present invention, in the step of obtaining the original image by executing step S411, the light signal is collected by using an image sensor to obtain the original image; and in the step of obtaining the original estimation result by executing step S412, the original estimation result is obtained by using a Poisson noise model and a Gaussian noise model according to the characteristic parameters of the original image and the image sensor.

[0106] Specifically, when executing step S412, the noise of the original image is composed of Poisson noise and Gaussian noise. One of the characteristic parameters of the image sensor is the global system gain K 全局 , the global system gain is an image sensor parameter that is easy to measure. In some embodiments of the present invention, K 全局 =5DN / e - DN is a dimensionless unit representing the grayscale value of a digital image, e - Represents unit charge, K 全局 Represents the grayscale value of the unit charge conversion output. According to the Poisson noise model, the Poisson noise variance of the original image is: K 全局 *E(x)≈K 全局 *μ x , where E(x) represents the mathematical expectation of the pixel value of any pixel in the original image, that is, the pixel value of the pixel when there is no noise. Since E(x) cannot be actually obtained, μ is used. x To approximate, μ x represents the average pixel value of all pixels in the small window around the pixel. It can be seen that Poisson noise is related to the average pixel value. xIt is easy to obtain from the original image, so the Poisson noise of the original image can be obtained. In addition, in addition to the Poisson noise, the noise of the original image also includes: the mean is 0, the standard deviation is σ G Gaussian white noise. G It can be calculated by characteristic parameters of the image sensor such as readout noise and dark current noise. In some embodiments of the present invention, σ G =3e - , where e - represents unit charge. Therefore, the Gaussian noise of the original image can be obtained. The total noise variance of the original image, i.e. the original estimation result, can be obtained by summing the Poisson noise variance and the Gaussian noise variance of the original image.

[0107] In the above-mentioned embodiment, the original estimation result is obtained based on the original image. However, this approach is only an example. In other embodiments of the present invention, the original estimation result can also be obtained based on the image to be estimated and the mapping curve.

[0108] refer to Figure 8 , shows a flow chart of the steps of obtaining the original estimation result in another embodiment of the noise estimation method of the present invention.

[0109] Specifically, the step of obtaining the original estimation result includes: firstly executing step S421 to obtain the inverse mapping curve according to the mapping curve of the nonlinear mapping. Figure 8 In the illustrated embodiment, the non-linear mapping is tone mapping.

[0110] Figure 8 In the embodiment shown, the image to be estimated I obtained after nonlinear mapping y Any pixel p y The pixel value is y, the original image I before nonlinear mapping x Any pixel p x The pixel value of is x; the mapping curve of the nonlinear mapping is: y=g(x). Therefore, in the step of performing step S421, in which the inverse mapping curve is obtained according to the mapping curve of the nonlinear mapping, the inverse mapping curve is: x=g r (y).

[0111] Specifically, Figure 8 In the illustrated embodiment, the tone mapping is implemented in a rational function transformation manner, that is, the mapping curve is:

[0112]

[0113] Among them, y max =2 10 -1,x max =2 14 -1, b=212 , a=512.

[0114] Therefore, in the step of executing step S421 to obtain the inverse mapping curve, the inverse mapping curve is:

[0115]

[0116] After the inverse mapping curve is obtained, step S422 is performed to obtain the original estimation result according to the inverse mapping curve and the image to be estimated.

[0117] Specific as Fig. 9 As shown, executing step S422, the step of obtaining the original estimation result includes: executing step S422a, obtaining the average pixel value of the pixel position to be inverted in the image to be estimated, wherein the average pixel value of the pixel position to be inverted is the average value of the pixel values ​​of multiple pixel positions in a preset inversion window, and the inversion window is centered on the pixel position to be inverted; executing step S422b, calculating the average pixel value of the corresponding pixel position in the original image according to the inverse mapping curve and the average pixel value of the pixel position to be inverted, wherein the corresponding pixel position is in the same position as the pixel position to be inverted; executing step S422c, obtaining the original estimation result according to the average pixel value of all pixel positions in the original image.

[0118] Specific as Figure 8 In the illustrated embodiment, executing step S422, the step of obtaining the original estimation result includes: y In the example, the pixel position p to be inferred is obtained y The average pixel value μ y , where the pixel to be inferred is p y The average pixel value is the average value of the pixel values ​​of multiple pixels in the preset reverse window, and the reverse window is based on the pixel position p to be reversed y As the center; According to the inverse mapping curve x = g r (y) and the pixel position to be inferred p y The average pixel value μ y , calculate the original image I x Inside, the corresponding pixel position p x The average pixel value μ x =g r (μ y ), where the corresponding pixel position p x and the pixel position p to be inferred y The same position; according to the original image I x The average pixel value μ of all pixels in x, and the characteristic parameters of the image sensor, based on the Poisson noise model and the Gaussian noise model, obtain the original estimation result var(x).

[0119] After obtaining the original estimation result var(x), the noise estimation result of the image to be estimated can be obtained in the same manner as in the above embodiment.

[0120] Combined with reference Fig.10 , showing Figure 8 The noise estimation result of the image to be estimated obtained by the noise estimation method embodiment shown in FIG. y , unit: DN; the vertical axis represents the noise variance σ of the image to be estimated y 2 , unit: DN 2 ; Circle data points represent Figure 8 The noise estimation result obtained by the noise estimation method embodiment is shown.

[0121] It should be noted that Figure 8 In the embodiment shown, the image to be estimated is a grayscale image, which is obtained by tone mapping the original image through rational function transformation. The original image is a grayscale image containing noise, and its brightness ranges from 0 to 2. 14 -1 is divided into 200 gray levels. After the original image is converted into the image to be estimated through tone mapping, the original image is no longer stored, and only the mapping curve of the nonlinear mapping (i.e. tone mapping) needs to be stored, thereby saving storage resources. x The relationship between can be calculated by the Poisson noise model and the Gaussian noise model, and the calculation method is the same as that in the above embodiment. Since the image to be estimated is a grayscale image, the actual noise on each grayscale can be easily counted by calculating the pixel value variance, so Fig.10 The figure also shows the noise result obtained according to the actual statistics of the image to be estimated, wherein the solid line represents the noise result obtained according to the actual statistics of the image to be estimated.

[0122] like Fig.10 As shown, the noise estimation result obtained by the noise estimation method embodiment of the present invention is highly consistent with the noise result obtained by the actual statistics of the image to be estimated, that is, the noise estimation method of the present invention can still obtain a fairly accurate noise estimation result. The reason for using a grayscale image in this embodiment is to facilitate the statistics of the actual noise of the image to be estimated to compare with the noise estimation result obtained by the noise estimation method of the present invention. In actual work, the noise estimation method of the present invention is not only applicable to grayscale images, but also to images of any scene.

[0123] In addition, each pixel of the original image is px There is a corresponding noise estimate σ x 2 Therefore, in the noise estimation method based on the original image, a large amount of line buffer is needed to store σ x 2 , so as to facilitate subsequent calculations.

[0124] and Figure 8 and Fig.10 In the embodiment of the noise estimation method shown in the figure, the noise estimation is performed directly based on the mapping curve and the image to be estimated, and the original image before nonlinear mapping is not required, and a high-accuracy noise estimation result can be obtained, which can save the storage of σ x 2 , delay, reading and other processes, thereby saving a lot of hardware storage resources, bandwidth and power consumption, which is beneficial to the implementation in hardware such as FPGA and IC.

[0125] It should be noted that Figure 1 In the illustrated embodiment, the original image 102 is a grayscale image, and the original estimation result can be obtained by actual statistics of the original image 102; the noise estimation result of the image to be estimated obtained by the embodiment of the noise estimation method of the present invention has a high consistency with the noise result actually obtained by testing the image to be estimated 101. In other embodiments of the present invention, for the image to be estimated, the noise estimation result obtained by the embodiment of the noise estimation method of the present invention has a fairly high consistency with the noise estimation result obtained by the random variable theory method.

[0126] Table 1 shows the noise estimation results obtained in some embodiments of the present invention and the noise estimation results obtained by the corresponding random variable theory method. The distribution parameters before mapping are assumed to be the original image with an average brightness equal to μ x The uniform image with mean 0 and variance σ is superimposed G 2 Gaussian noise, each image to be estimated is obtained by performing the same nonlinear mapping on the corresponding original image, wherein the nonlinear mapping is tone mapping, and the tone mapping is implemented in the form of rational function transformation, and the mapping curve of the tone mapping is Figure 4 The mapping curve of a=512.

[0127] Table 1: Noise estimation results obtained by multiple embodiments of the noise estimation method of the present invention and multiple noise estimation results obtained by the corresponding random variable theory method

[0128]

[0129] The random variable theory method is the method introduced above, which will not be described here. The following uses the distribution parameter data before mapping with sequence number 1 in Table 1 to illustrate how to obtain the result of the noise estimation method of the present invention. It is easy to know that the original estimation result is σ G 2 =1000, according to the formula of rational function transformation:

[0130]

[0131] Among them, y max =2 10 -1,x max =2 14 -1, b=2 12 , a=512. It is easy to see that when x=μ x =200 when the first-order derivative value is:

[0132]

[0133] Therefore, the noise variance obtained by the noise estimation method of the present invention is σ y 2 =0.4244 2 *1000=180.1.

[0134] The same method can be used to obtain noise estimation results with serial numbers 2 to 4, as shown in Table 1. The noise estimation results obtained by the noise estimation method of the present invention are consistent with the noise estimation results obtained by the random variable theory method, and have high consistency.

[0135] Combined with reference Figures 11 to 14 , which shows the noise estimation results obtained by multiple embodiments of the noise estimation method of the present invention.

[0136] It should be noted that Figures 11 to 14 In the illustrated embodiment, the images to be estimated with different noise estimation results are images obtained after the same original image is subjected to different nonlinear mappings; wherein the nonlinear mappings are all tone mappings, and the tone mappings are all implemented in the form of rational function transformations, and the mapping curve of the tone mapping is Figure 4 The mapping curve shown in .

[0137] In addition, to facilitate comparison, Figures 11 to 14 The noise estimation result obtained by the random variable theory method for the image to be estimated is also shown.

[0138] As shown in the figure, the horizontal axis represents the mean brightness μ Y , unit: DN; the vertical axis represents the noise variance σ Y 2 , unit: DN 2; The hollow circles represent the noise estimation results obtained by the noise estimation method of the present invention; the solid lines represent the noise estimation results obtained by the random variable theory method; Figures 11 to 14 The mapping curves are shown respectively. Figure 4 The noise estimation results when a=256, 512, 1024, and 2048 in the mapping curve shown.

[0139] like Figures 11 to 14 As shown, for the image to be estimated obtained after the rational function transformation, the calculation results of the random variable theory method are compared with the calculation results of the image noise estimation method of the present invention, and the two are highly consistent and have a high degree of conformity.

[0140] Table 2: Figures 11 to 14 The time taken to perform the calculations for the multiple noise estimation results is shown.

[0141]

[0142] Table 2 shows the different methods to obtain Figures 11 to 14 The time taken to perform the calculation of the noise estimation result is shown. It can be seen that the time taken to perform the calculation of the noise estimation method of the present invention is only about 1 / 1000 of the time taken by the random variable theory method, that is, the calculation speed of the noise estimation method of the present invention is about 1000 times faster than that of the random variable theory method.

[0143] In addition, the random variable theory method uses the formula D(X) = E(X 2 )-E 2 (X), where, relative to D(X), E(X 2 ) and E 2 (X) is several orders of magnitude larger, Figures 11 to 14 In the embodiment shown, E(X 2 ) and E 2 (X) is 10 4 Up to 10 5 ; while D(X) is only about 10 2 , so E(X 2 ) and E 2 The acquisition of (X) requires high calculation accuracy. Even a one thousandth error will lead to a large error in the obtained D(X), as shown in Table 3. Where D(X) represents the variance of the random variable X, and E(X) represents the mathematical expectation of the random variable X.

[0144] Table 3: E(X) obtained by random variable theory method in the process of noise estimation of two images to be estimated 2 )、E 2 (X) and D(X)

[0145] E(X2) E(X) D(X) 290620.84 538.896 211.9 290620.84 539.2 -115.8

[0146] As shown in the table, E(X) has changed by less than 1 / 1000, but D(X) has changed from a reasonable estimate to a meaningless negative value.

[0147] In the noise estimation method of the present invention, the noise estimation result is obtained based on the mapping derivative result of the mapping curve and the original estimation result of the original image, and does not involve E(X 2 )、E 2 The calculation of (X) and D(X) can not only effectively reduce the computational load, but also has stronger robustness and stable calculation results.

[0148] In the above-mentioned embodiment, the mapping curve is an explicit analytical function, and the function formula of the mapping curve can be directly derived to obtain the mapping derivative result. In other embodiments of the present invention, the mapping relationship between the original image and the image to be estimated can also be obtained based on the discretized mapping curve, thereby obtaining the mapping derivative result.

[0149] refer to Fig.15 , shows the noise estimation result obtained in another embodiment of the noise estimation method of the present invention.

[0150] Due to the limitation of hardware resources, Fig.15 In the embodiment shown, the nonlinear mapping of the image to be estimated is achieved by a lookup table (LUT), so in the step of obtaining the mapping curve of the nonlinear mapping, the mapping curve is a discretized mapping curve. Specifically, the nonlinear mapping is a tone mapping from 14 bits to 10 bits, and the tone mapping is achieved by rational function transformation. The parameters of the mapping curve of the tone mapping are: x max =2 14 ,y max =2 10 , b=2 12 , a=2 9 .

[0151] Therefore, the step of obtaining the mapping curve of the nonlinear mapping includes: discretizing the mapping curve of the nonlinear mapping. Fig.15 In the illustrated embodiment, the step of discretizing the mapping curve of the nonlinear mapping includes: taking 0 to x max A plurality of x values ​​with equal spacing within the interval are brought into the mapping curve of the nonlinear mapping to obtain a table to be searched.

[0152] Specifically, take 0 to x max =2 14The 129 x values ​​with equal spacing in the interval have 128 uniform intervals, that is, x=0, 128, 256, ..., 16384; the 129 x values ​​are substituted into the mapping curve of the rational function transformation from 14 bits to 10 bits to obtain a table to be searched consisting of 129 values.

[0153] In the step of obtaining the image to be estimated, the linear original image is mapped to the nonlinear image to be estimated according to the table to be searched; due to the size limitation of the table to be searched, before mapping, the step of obtaining the image to be estimated further includes: interpolating the table to be searched. Fig.15 In the illustrated embodiment, in the step of interpolating the table to be searched, the table to be searched is interpolated by linear interpolation.

[0154] It should be noted that, since the mapping curve of the nonlinear mapping is discretized, in the step of obtaining the mapping derivative result according to the mapping curve, the mapping derivative result is obtained by difference. Fig.15 In the illustrated embodiment, the mapping derivative result is obtained by difference according to the table to be looked up after interpolation.

[0155] like Fig.15 As shown, the horizontal axis represents the brightness mean μ of the image to be estimated, unit: DN; the vertical axis represents the noise variance var of the image to be estimated, unit: DN 2 ; The hollow circles represent the noise estimation results obtained by the noise estimation method of the present invention.

[0156] It should be noted that Fig.15 In the illustrated embodiment, the original image is a grayscale image containing noise; therefore Fig.15 The noise result obtained based on the actual statistics of the image to be estimated is also shown, where the solid line represents the noise result obtained based on the actual statistics of the image to be estimated. Fig.15 It can be seen that the noise estimation result obtained by the noise estimation method embodiment of the present invention is consistent with the noise result obtained by actual statistics of the image to be estimated, that is, the table to be searched composed of 129 values, combined with simple linear interpolation, can obtain a noise estimation result with higher accuracy.

[0157] refer to Fig.16 , shows the noise estimation result obtained in another embodiment of the noise estimation method of the present invention.

[0158] Fig.16 Illustrated embodiment Fig.15 The same points as the embodiments shown are not described in detail in the present invention; Fig.16 The embodiment shown and Fig.15 The embodiment shown differs in that: Fig.16 In the illustrated embodiment, in the step of interpolating the table to be searched, the table to be searched is interpolated by cubic interpolation.

[0159] akin, Fig.16 In the illustrated embodiment, the original image is also a grayscale image containing noise. Fig.16 The figure also shows the noise result obtained according to the actual statistics of the image to be estimated, wherein the solid line represents the noise result obtained according to the actual statistics of the image to be estimated.

[0160] Depend on Fig.16 It can be seen that the noise estimation result obtained by the embodiment of the noise estimation method of the present invention is more consistent with the noise result obtained by the actual statistics of the image to be estimated, that is, the higher the interpolation accuracy in the step of interpolating the table to be looked up, the higher the accuracy of the noise estimation result obtained by the embodiment of the noise estimation method of the present invention; therefore, by improving the interpolation accuracy of the table to be looked up, the accuracy of the noise estimation result obtained by the embodiment of the noise estimation method of the present invention can be further improved.

[0161] It should be noted that in the above embodiments, the mapping curves are all mapping curves based on display analytical functions. However, this approach is only an example, and in other embodiments of the present invention, the mapping curves can also be generated based on actual images.

[0162] It should also be noted that Fig.16 In the illustrated embodiment, the mapping curve of the nonlinear mapping is also discretized, so in the step of obtaining the mapping derivative result, the mapping derivative result is also obtained by difference. Specifically, the mapping derivative result is obtained by difference according to the interpolated table to be searched.

[0163] Compare Fig.15 and Fig.16 , it can be seen that in the process of interpolating the table to be searched, the interpolation accuracy not only affects the mapping relationship of the pixel values, but also affects the accuracy of the mapping derivative results obtained. Since the cubic interpolation method with higher accuracy is used to interpolate the table to be searched, Fig.16 The noise estimation result obtained by the illustrated embodiment has a higher consistency with the actual statistical result.

[0164] refer to Fig.17 and Fig.18 ,in Fig.17 The mapping curve of the nonlinear mapping in another embodiment of the noise estimation method of the present invention is shown, which is composed of a lookup table consisting of 129 values; Fig.18 Shows Fig.17 The noise estimation result obtained in the noise estimation method embodiment is shown.

[0165] Fig.17 In FIG. 1 , the horizontal axis represents the numbers of the 129 values ​​in the lookup table, and the vertical axis represents the pixel values ​​of the image to be estimated.

[0166] In some embodiments of the present invention, in the step of obtaining the image to be estimated, the nonlinear mapping is performed by a histogram equalization method. Histogram equalization is a method for implementing nonlinear mapping such as tone mapping, dynamic range compression, and contrast adjustment. Fig.17 and Fig.18 In the illustrated embodiment, the nonlinear mapping is tone mapping achieved by a contrast-limited histogram equalization method.

[0167] Contrast Limited Histogram Equalization (CLHE) is a popular pixel value nonlinear mapping method, which is often used in image contrast stretching or tone mapping. Since contrast limited histogram equalization adopts the cumulative histogram (CDF) approach, the mapping curve generated by the mapping method of contrast limited histogram equalization is monotonic and first-order differentiable. Fig.17 In the illustrated embodiment, the mapping curve of the nonlinear mapping is a mapping curve generated by a contrast-limited histogram equalization mapping method.

[0168] It should be noted that Fig.17 In the illustrated embodiment, the nonlinear mapping is a tone mapping of 14 bits to 10 bits, and the tone mapping is implemented by a look-up table (LUT), wherein the table to be looked up consists of 129 data.

[0169] It should also be noted that Fig.17 In the illustrated embodiment, the mapping curve is a mapping curve obtained based on a preset image. In other embodiments of the present invention, the mapping curve may also be obtained based on the image to be estimated. The technical solution of the present invention does not limit the method for obtaining the mapping curve.

[0170] also, Fig.17 In the embodiment shown, the image to be estimated is an original image Fig.17 The curve shown is an image obtained after tone mapping of the mapping curve; specifically, the original image is a grayscale image containing noise (such as Figure 3 shown).

[0171] like Fig.18As shown, the horizontal axis represents the brightness mean μ of the image to be estimated, unit: DN; the vertical axis represents the noise variance var of the image to be estimated, unit: DN 2 ; The hollow circles represent the noise estimation results obtained by the noise estimation method of the present invention.

[0172] It should be noted that Fig.18 In the illustrated embodiment, the original image is a grayscale image containing noise, so the image to be estimated after tone mapping is also a grayscale image; therefore Fig.18 The figure also shows the noise result obtained according to the actual statistics of the image to be estimated, wherein the solid line represents the noise result obtained according to the actual statistics of the image to be estimated.

[0173] Depend on Fig.18 It can be seen that the noise estimation result obtained by the embodiment of the noise estimation method of the present invention is consistent with the noise result obtained by the actual statistics of the image to be estimated. It can be seen that when the mapping curve of the nonlinear mapping is a mapping method based on image generation with contrast-limited histogram equalization, the noise estimation method of the present invention can still achieve noise estimation quickly and accurately.

[0174] Fig.19 The figure shows the noise estimation result obtained in another embodiment of the noise estimation method of the present invention.

[0175] It should be noted that Fig.19 In the embodiment, in the step of obtaining the image to be estimated, the nonlinear mapping is tone mapping, and the mapping curve of the tone mapping is Fig.17 and Fig.18 The mapping curves of the tone mapping in the illustrated embodiment are the same; and the nonlinear mapping is also a 14-bit to 10-bit tone mapping, and is implemented by a lookup table (LUT), wherein the table to be looked up consists of 129 data; in addition, Fig.19 In the embodiment shown, the original image is also a grayscale image containing noise (such as Figure 3 shown).

[0176] Fig.19 The embodiment shown and Fig.18 The embodiment shown differs in that Fig.19 In the illustrated embodiment, in the step of interpolating the table to be searched, the table to be searched is interpolated by cubic interpolation.

[0177] Similarly, due to Fig.19 In the illustrated embodiment, the original image is also a grayscale image containing noise, so the image to be estimated after nonlinear mapping is also a grayscale image. Fig.19The figure also shows the noise result obtained according to the actual statistics of the image to be estimated, wherein the solid line represents the noise result obtained according to the actual statistics of the image to be estimated.

[0178] Depend on Fig.19 It can be seen that the noise estimation result obtained by the embodiment of the noise estimation method of the present invention is highly consistent with the noise result obtained by actual statistics of the image to be estimated.

[0179] It should be noted that in the above embodiments, the original images are all grayscale images containing noise. In other embodiments of the present invention, the original images may also be images of any scene actually captured by a camera, that is, images actually obtained by an image sensor.

[0180] refer to Figure 20 to Figure 25 , shows the images, curves and results obtained in another embodiment of the noise estimation method of the present invention, wherein, Fig. 20 The image to be estimated obtained by the noise estimation method embodiment of the present invention is shown; Fig.21 Shows Fig. 20 The original image in the embodiment of the noise estimation method shown; Fig. 22 Shows Fig. 20 A mapping curve of nonlinear mapping in the noise estimation method embodiment shown; Fig.23 Shows Fig. 20 and Fig.21 In the embodiment of the noise estimation method shown, Fig. 22 The mapping derivative results obtained by the mapping curve shown; Fig.24 The noise estimation result of the original image is shown; Fig.25 Shows Fig. 22 and Fig.21 The noise estimation result obtained by the noise estimation method embodiment shown,

[0181] like Fig.21 As shown, the original image is an image obtained by actually shooting a color card with grayscale with a black and white digital camera; the nonlinear mapping is tone mapping, that is, the image to be estimated is obtained by tone mapping the original image.

[0182] like Fig. 22 The mapping curve shown is a mapping method using contrast-limited histogram equalization based on Fig.21 The mapping curve generated from the original image is shown.

[0183] like Fig. 22As shown, the horizontal axis represents the pixel value of the pixel in the original image, unit: DN, and the vertical axis represents the pixel value of the pixel in the image to be estimated, unit: DN. As can be seen from the figure, the mapping curve is monotonic, that is, when the pixel value of the pixel in the original image increases or decreases, the pixel value of the pixel in the image to be estimated increases or decreases accordingly. Specifically, the mapping curve is monotonically increasing.

[0184] Fig.23 The mapping derivative result shown is the first-order derivative of the mapping curve.

[0185] Since the original image is an image obtained by actually photographing a color card with grayscale using a black and white digital camera, the original estimation result can be obtained based on the pixel values ​​of the white and gray blocks below the color card in the original image. Fig.24 As shown, the horizontal axis represents the average pixel value, unit: DN; the vertical axis represents the noise variance, unit: DN 2 .

[0186] Fig.25 As shown in the figure, the horizontal axis represents the average pixel value, unit: DN; the vertical axis represents the noise variance, unit: DN 2 ; The circle data points represent the noise estimation results obtained by the noise estimation method embodiment; the cross-shaped data points represent the noise results obtained according to the statistics of the image to be estimated.

[0187] As shown in the figure, the noise estimation result obtained by the embodiment of the noise estimation method is highly consistent with the noise result obtained by actual testing in the image to be estimated. That is to say, the noise estimation method of the present invention can also obtain high-precision noise estimation results for images directly obtained by camera photography.

[0188] Correspondingly, the present invention also provides a noise estimation device, comprising: a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the processor executes the steps of the noise estimation method of the present invention.

[0189] and a storage medium, wherein the storage medium is a non-volatile storage medium or a non-transient storage medium, on which computer instructions are stored, and when the computer instructions are executed, the steps of the noise estimation method of the present invention are executed.

[0190] Since the processor of the noise estimation device is used to execute the steps of the noise estimation method of the present invention; and the computer instructions stored in the storage medium execute the steps of the noise estimation method of the present invention when running, the specific technical solutions of the noise estimation device and the storage medium refer to the embodiments of the noise estimation method mentioned above, and the present invention will not be repeated here.

[0191] In addition, the present invention also provides an image processing method, comprising: performing noise estimation and obtaining a noise estimation result, wherein in the step of performing noise estimation, noise estimation is performed by the noise estimation method of the present invention.

[0192] In some embodiments of the present invention, noise reduction processing is performed according to the noise estimation result. The noise estimation method can accurately estimate the noise of the image to be estimated, and the noise reduction processing performs noise reduction according to the noise estimation result, so the noise reduction effect is good, and the noise points can be effectively reduced and the image quality can be improved; and the noise estimation method performs noise estimation on the image to be estimated obtained after nonlinear mapping based on the mapping curve of the nonlinear mapping and the original estimation result, which can effectively save hardware resources and have high accuracy, and can effectively ensure the effect of noise reduction processing.

[0193] Correspondingly, the present invention also provides an image processing device, comprising: a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the processor executes the steps of the image processing method of the present invention.

[0194] A storage medium is provided, wherein the storage medium is a non-volatile storage medium or a non-transient storage medium, on which computer instructions are stored, and when the computer instructions are executed, the steps of the image processing method of the present invention are executed.

[0195] Since the processor of the image processing device is used to execute the steps of the image processing method of the present invention; the computer instructions stored in the storage medium execute the steps of the image processing method of the present invention when running, the specific technical solutions of the image processing device and the storage medium refer to the embodiments of the aforementioned image processing method, and the present invention will not be repeated here.

[0196] In summary, according to the mapping curve, a mapping derivative result is obtained; an original estimation result is obtained, and the original estimation result is a noise estimation result of the original image; according to the mapping derivative result and the original estimation result, a noise estimation result of the image to be estimated is obtained. The image to be estimated obtained after nonlinear mapping is subjected to noise estimation, so as to perform subsequent noise reduction processing, etc., which can effectively save hardware resources, and the noise estimation result is obtained based on the mapping derivative result and the original estimation result related to the mapping curve of the nonlinear mapping, so the noise estimation result is related to the nonlinear mapping, and the noise estimation result has higher accuracy, which can effectively improve the effect of subsequent noise reduction processing, and only needs to be derived for the mapping curve of the nonlinear mapping, so the calculation amount is small and the calculation speed is fast.

[0197] Moreover, according to the mapping curve of the nonlinear mapping, an inverse mapping curve is obtained; according to the inverse mapping curve, in combination with the image to be estimated, the original estimation result is obtained. Based on the inverse mapping curve of the mapping curve, in combination with the image to be estimated, the original estimation result of the original image is inferred, which can save a lot of storage data, delay and reading process, can effectively save hardware resources such as storage, bandwidth and power consumption, and is more conducive to implementation on hardware platforms such as FPGA.

[0198] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the claims.

Claims

1. A noise estimation method, characterized in that: include: Obtaining an image to be estimated, wherein the image to be estimated is an image obtained after nonlinear mapping is performed on the original image, wherein the nonlinear mapping is a mapping for pixel values; Obtaining a mapping curve of the nonlinear mapping, wherein the mapping curve is a curve that is at least first-order differentiable; According to the mapping curve, obtaining a mapping derivative result; Obtaining an original estimation result, where the original estimation result is a noise estimation result of the original image; Obtaining a noise estimation result of the image to be estimated according to the mapping derivative result and the original estimation result, including: obtaining the noise estimation result of the image to be estimated according to the product of the square of the mapping derivative result and the original estimation result.

2. The noise estimation method according to claim 1, characterized in that: In the step of obtaining a mapping curve of the nonlinear mapping, the mapping curve is monotonic.

3. The noise estimation method according to claim 1, characterized in that: In the step of obtaining the mapping derivative result, a first-order derivative is performed on the mapping curve of the nonlinear mapping.

4. The noise estimation method according to claim 1, characterized in that: In the step of obtaining the image to be estimated, the nonlinear mapping includes one of tone mapping, gamma correction, dynamic range compression and contrast adjustment.

5. The noise estimation method according to claim 1 or 4, characterized in that: In the step of obtaining the image to be estimated, the nonlinear mapping is performed by a histogram equalization method.

6. The noise estimation method according to claim 1, characterized in that: In the step of obtaining the noise estimation result of the image to be estimated according to the mapping derivative result and the original estimation result, the noise variance of the image to be estimated is: ； in, represents the noise variance of any pixel (i, j) of the image to be estimated, σ x 2 (i, j) represents the noise variance of the same pixel position (i, j) of the original image, g'(μ) represents the derivative result of the mapping, and μ represents the average pixel value in a small area near the pixel position (i, j) in the original image.

7. The noise estimation method according to claim 1, characterized in that: The steps to obtain the original estimate are: Get the original image; The original estimation result is obtained according to the original image.

8. The noise estimation method according to claim 7, characterized in that: In the step of obtaining the original image, an image sensor is used to collect light signals to obtain the original image; In the step of obtaining the original estimation result, the original estimation result is obtained according to the original image and characteristic parameters of the image sensor through a Poisson noise model and a Gaussian noise model.

9. The noise estimation method according to claim 1, characterized in that: The steps to obtain the original estimate are: Obtaining an inverse mapping curve according to the mapping curve of the nonlinear mapping; The original estimation result is obtained according to the inverse mapping curve and in combination with the image to be estimated.

10. The noise estimation method according to claim 9, characterized in that: The steps of obtaining the original estimation result include: In the image to be estimated, an average pixel value of the pixel to be inverted is obtained, wherein the average pixel value of the pixel to be inverted is an average value of pixel values ​​of multiple pixels in a preset inverted window, and the inverted window is centered on the pixel to be inverted; Calculate the average pixel value of the corresponding pixel in the original image according to the inverse mapping curve and the average pixel value of the pixel to be inferred, wherein the corresponding pixel is at the same position as the pixel to be inferred; The original estimation result is obtained according to the average pixel value of all pixel positions in the original image.

11. A noise estimation device, comprising: A memory and a processor, wherein the memory stores a computer program that can be run on the processor, wherein: When the processor executes the program, the processor performs the steps of the noise estimation method according to any one of claims 1 to 10.

12. A storage medium, wherein the storage medium is a non-volatile storage medium or a non-transient storage medium, and a computer instruction is stored thereon, wherein: When the computer instructions are executed, the steps of the noise estimation method according to any one of claims 1 to 10 are executed.

13. An image processing method, characterized in that: include: Noise estimation is performed to obtain a noise estimation result. In the step of performing noise estimation, noise estimation is performed by the noise estimation method according to any one of claims 1 to 10.

14. The image processing method according to claim 13, characterized in that: Also includes: Noise reduction processing is performed according to the noise estimation result.

15. An image processing device, comprising: A memory and a processor, wherein the memory stores a computer program that can be run on the processor, wherein: When the processor executes the program, the processor performs the steps of the image processing method according to any one of claims 13 to 14.

16. A storage medium, wherein the storage medium is a non-volatile storage medium or a non-transient storage medium, and a computer instruction is stored thereon, wherein: When the computer instructions are executed, the steps of the image processing method according to any one of claims 13 to 14 are executed.

Images