Image processing method and device, electronic equipment and storage medium

Abstract

The invention discloses an image processing method and device, electronic equipment and a storage medium, and relates to the technical field of image processing. The method comprises the steps of obtaining a to-be-processed image; determining a pixel attribution value of a current pixel point in the to-be-processed image; determining a pixel smooth value of a current pixel point in the to-be-processed image; according to the pixel attribution value and the pixel smooth value of the current pixel point, calculating a local variance value of the current pixel point according to a preset condition; and based on the brightness value of each pixel point in the to-be-processed image, performing segmented statistics on the local variance value corresponding to each pixel point to obtain noise level estimation of the HDR image in different brightness intervals. A pixel source is determined by calculating a pixel attribution value for an HDR image, interference of a texture area is avoided by calculating a pixel smooth value, and then a local variance value is calculated to reflect the noise level of a pixel point, so that the estimation precision of noise is improved. And finally, noise level estimation under different brightness intervals is output, and high-quality input is provided for subsequent image processing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an image processing method and device, electronic equipment and storage medium. BACKGROUND

[0002] Image denoising technology is one of the important topics in ISP (Image Signal Processor) technology. A common denoising method is to obtain all pixels in the neighborhood of the current pixel, low-pass filter the pixels to obtain the denoised pixels. The above denoising method usually reduces the high frequency part of the image, making the image blurred.

[0003] In the conventional image denoising processing, the noise level NStd and the high frequency component HF of the current image are first measured. Then, by comparing the sizes of the two, it is determined whether the current image region is image texture. For example, when HF is significantly higher than NStd, it is determined that the current image region is image texture, so as to avoid the blurring processing of the image by denoising. When HF is approximately equal to or less than NStd, it is determined that the current image region is a noise region, and the image is subjected to strong blurring processing.

[0004] However, the current scheme can estimate the noise of some conventional images, but the HDR image is usually fused from two or more RAW images, and the noise levels of the RAW images before fusion are different. Since the image signal is amplified in the fusion process of the HDR, the noise level after fusion is often discontinuous, and there are faults at the junction of image fusion, resulting in that there is no good solution for noise estimation of the HDR image at present. SUMMARY

[0005] Therefore, the present application provides an image processing method and device, electronic equipment and storage medium, which mainly aims to solve the problem that the noise of the HDR image cannot be estimated at present.

[0006] In a first aspect, the present application provides an image processing method, comprising: obtaining a to-be-processed image, the to-be-processed image being an HDR image fused from at least two original images; determining a pixel attribution value of a current pixel point in the to-be-processed image, the pixel attribution value being used to determine the original image to which the current pixel point belongs; determining a pixel smoothing value of the current pixel point in the to-be-processed image, the pixel smoothing value being used to determine the image smoothing degree of the region where the current pixel point is located; calculating a local variance value of the current pixel point according to the pixel attribution value and the pixel smoothing value of the current pixel point under a preset condition; Based on the brightness value of each pixel point in the image to be processed, the local variance value corresponding to each pixel point is segmented and counted to obtain the noise level estimation of the HDR image under different brightness intervals.

[0007] Optionally, the determining of the pixel attribution value of the current pixel point in the image to be processed comprises: acquiring the brightness value of each original image fused to obtain the image to be processed; acquiring the brightness value of the current pixel point; and determining the pixel attribution value of the current pixel point according to the brightness value of the current pixel point and the brightness value of each original image.

[0008] Optionally, the determining of the pixel attribution value of the current pixel point according to the brightness value of the current pixel point and the brightness value of each original image comprises: if the image to be processed is fused from a first original image and a second original image, acquiring the brightness value of the current pixel point; comparing the brightness value of the current pixel point with a preset brightness threshold, and assigning a first attribution value to the current pixel point in the case that the brightness value of the current pixel point is less than or equal to the preset brightness threshold, or assigning a second attribution value to the current pixel point in the case that the brightness value of the current pixel point is greater than the preset brightness threshold; wherein the first attribution value represents that the current pixel point is derived from the first original image, the second attribution value represents that the current pixel point is derived from the second original image, and the preset brightness threshold is set according to the brightness value of the first original image corresponding to the fusion and the brightness value of the second original image corresponding to the fusion.

[0009] Optionally, the determining of the pixel smoothing value of the current pixel point in the image to be processed comprises: acquiring the pixel value of each pixel point in a first preset region with the current pixel point as the center; performing filtering calculation on the pixel value of each pixel point in the first preset region through a high-pass filtering operator to obtain a high-frequency component corresponding to the first preset region; comparing the high-frequency component corresponding to the first preset region with a preset component threshold, and assigning a first smoothing value to the current pixel point in the case that the high-frequency component corresponding to the first preset region is less than or equal to the preset component threshold, or assigning a second smoothing value to the current pixel point in the case that the high-frequency component corresponding to the first preset region is greater than the preset component threshold; wherein the first smoothing value represents that the region where the current pixel point is located is a flat region, and the second smoothing value represents that the region where the current pixel point is located is a texture region, and the high-pass filtering operator comprises a Sobel operator or a Laplace operator.

[0010] Optionally, the calculating the local variance value of the current pixel point according to the pixel attribution value and the pixel smoothing value of the current pixel point under a preset condition comprises: taking the current pixel point as a center, filtering out a first pixel point set with the pixel smoothing value being the second smoothing value in a second preset region; determining the pixel attribution value of the current pixel point, filtering out pixel points different from the current pixel point in the first pixel point set to obtain a second pixel point set; and calculating the local variance value of the current pixel point according to the current pixel point and the second pixel point set.

[0011] Optionally, the calculating the local variance value of the current pixel point according to the current pixel point and the second pixel point set is calculated by using the following formula: wherein, is the local variance value of the current pixel point, is a pixel value corresponding to an i-th pixel point in the second preset region; is an attribution judgment, in a case that the i-th pixel point and the current pixel point have the same pixel attribution value, is 1, otherwise 0; is a smoothing judgment, in a case that the pixel smoothing value of the current pixel point is the first smoothing value, is 1, otherwise 0.

[0012] Optionally, the segmenting and counting the local variance value corresponding to each pixel point based on the brightness value of each pixel point in the to-be-processed image to obtain the noise level estimation of the HDR image under different brightness intervals comprises: dividing the brightness of the to-be-processed image into N continuous brightness intervals, wherein N is an integer greater than 1; according to the brightness value of each pixel point in the to-be-processed image, dividing each pixel point and the corresponding local variance value into the corresponding brightness interval; for each brightness interval, calculating a statistical average value of the local variance value corresponding to all pixel points falling into the interval; taking the statistical average value corresponding to each brightness interval as the noise level estimation value of the HDR image in the brightness interval.

[0013] In a second aspect, the present application provides an image processing device, comprising: an acquisition unit configured to acquire a to-be-processed image, the to-be-processed image being an HDR image obtained by fusing at least two original images; a first calculation unit configured to determine a pixel attribution value of a current pixel point in the to-be-processed image, the pixel attribution value being used to judge an original image to which the current pixel point belongs; a second calculation unit configured to determine a pixel smoothness value of the current pixel point in the image to be processed, the pixel smoothness value being used to determine a smoothness degree of an image region where the current pixel point is located; a processing unit configured to calculate a local variance value of the current pixel point according to the pixel attribution value and the pixel smoothness value of the current pixel point under a preset condition; a statistical unit configured to perform piecewise statistics on the local variance value corresponding to each pixel point based on a luminance value of each pixel point in the image to be processed, to obtain a noise level estimation of the HDR image under different luminance intervals.

[0014] In a third aspect, the present application provides a computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the image processing method of the first aspect.

[0015] In a fourth aspect, the present application provides an electronic device including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, the processor implementing the image processing method of the first aspect when executing the computer program.

[0016] By means of the above technical solution, the image processing method, device, electronic device and storage medium provided by the present application first acquire an image to be processed, which is an HDR image obtained by fusing at least two original images; determine a pixel attribution value of a current pixel point in the image to be processed, the pixel attribution value being used to determine an original image to which the current pixel point belongs; determine a pixel smoothness value of the current pixel point in the image to be processed, the pixel smoothness value being used to determine a smoothness degree of an image region where the current pixel point is located; calculate a local variance value of the current pixel point according to the pixel attribution value and the pixel smoothness value of the current pixel point under a preset condition; perform piecewise statistics on the local variance value corresponding to each pixel point based on a luminance value of each pixel point in the image to be processed, to obtain a noise level estimation of the HDR image under different luminance intervals. The present application proposes a method for estimating noise of an HDR image, determines the pixel source by calculating the pixel attribution value of each pixel point of the HDR image, avoids the interference of a texture region by calculating the pixel smoothness value, and then calculates the local variance value based on the pixel attribution value and the pixel smoothness value to reflect the noise level of the pixel point, thereby improving the estimation accuracy of the noise. Finally, the noise level estimation under different luminance intervals is output, providing a high-quality input for subsequent image processing.

[0017] The above description is only a summary of the technical solutions of the present application. In order to more clearly understand the technical means of the present application, the specific embodiments of the present application can be implemented in accordance with the content of the description, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings required by the embodiments or prior art description will be briefly introduced as follows. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative labor.

[0020] Figure 1 A flowchart of an image processing method provided by an embodiment of the present application is shown; Figure 2 A schematic diagram of image fusion according to scene brightness changes provided by an embodiment of the present application is shown; Figure 3 A schematic diagram of a noise level estimation curve provided by an embodiment of the present application is shown; Figure 4 A flowchart of another image processing method provided by an embodiment of the present application is shown; Figure 5 A structural schematic diagram of an image processing device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0021] In order to more clearly illustrate the above-mentioned purposes, features and advantages of the present application, the solutions of the present application will be further described below. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0022] An image processing method proposed in the present embodiment is applied to an image processing device or an electronic device, which can be installed or integrated in some image noise analysis system or image noise reduction processing system, and can execute any of the image processing methods mentioned below when running.

[0023] In order to improve the problem that the noise estimation accuracy of the fused HDR image is poor and there is no good noise estimation for the HDR image. The present embodiment proposes an image processing method, as shown in Figure 1 The method comprises: S101, acquiring a to-be-processed image.

[0024] The image to be processed is an HDR image obtained by fusing at least two original images. For example, two or more initial RAW images with different initial brightness are enlarged by a certain scaling factor, and then fused to obtain an HDR image.

[0025] Here, taking the fusion of two RAW images as an example, the fusion process is briefly introduced. An image sensor captures a scene, thereby converting a light signal into an electrical signal to obtain an image. Each image has a cutoff signal, i.e., the maximum brightness that the image can carry. Because the exposure times of different RAW images are different, the maximum brightness and noise they present are also different. When fusing, the RAW image with lower brightness is enlarged, and then fused with another image. In this process, the brightness and noise level of the fused image are often discontinuous, and a fault occurs at the junction of the image fusion.

[0026] S102, determine the pixel attribution value of the current pixel point in the image to be processed.

[0027] The pixel attribution value is used to determine the original image to which the current pixel point belongs, i.e., to distinguish which original image the current pixel point belongs to among the original two or more original images. The role of the distinction is that in the subsequent local variance value calculation, the pixel points from the same original image are taken as one of the preset conditions, thereby avoiding the interference of other image pixel points after fusion, and thereby improving the accuracy of the local variance value for reflecting the noise level of the current pixel point.

[0028] S103, determine the pixel smoothing value of the current pixel point in the image to be processed.

[0029] The pixel smoothing value is used to determine the image smoothing degree of the region where the current pixel point is located. The pixel smoothing degree refers to the smoothing degree of the region where the current pixel point is located, i.e., the smoothing degree of all pixel points in a preset range or window centered on the current pixel point, thereby determining whether this region is smooth or textured.

[0030] S104, according to the pixel attribution value and the pixel smoothing value of the current pixel point, calculate the local variance value of the current pixel point according to the preset condition.

[0031] The local variance value of the current pixel point reflects the noise level of the current pixel point. Because the pixel attribution value and the pixel smoothing value of the current pixel point are introduced, the evaluation accuracy of the local variance value of the current pixel point can be improved by adjusting the preset condition to avoid the interference of non-identical images and textured regions.

[0032] S105, based on the brightness value of each pixel point in the image to be processed, the local variance value corresponding to each pixel point is segmented and counted to obtain the noise level estimation of the HDR image under different brightness intervals.

[0033] Since the noise level depends on the intensity of the signal, that is, the brightness of the pixel. Therefore, according to the brightness, the noise level estimation of the HDR image under different brightness intervals is obtained, such as the noise at brightness 100 and the noise at brightness 800, so as to facilitate the application and processing of other noise reduction algorithms.

[0034] In this embodiment, first, the image to be processed is obtained, which is an HDR image obtained by fusing at least two original images; the pixel attribution value of the current pixel point in the image to be processed is determined, which is used to determine the original image to which the current pixel point belongs; the pixel smoothing value of the current pixel point in the image to be processed is determined, which is used to determine the image smoothing degree of the region where the current pixel point is located; the local variance value of the current pixel point is calculated according to the pixel attribution value and the pixel smoothing value of the current pixel point; based on the brightness value of each pixel point in the image to be processed, the local variance value corresponding to each pixel point is segmented and counted to obtain the noise level estimation of the HDR image under different brightness intervals. The present application proposes a method for noise estimation of an HDR image, which determines the pixel source by calculating the pixel attribution value of each pixel point of the HDR image, avoids the interference of the texture region by calculating the pixel smoothing value, and then calculates the local variance value based on the pixel attribution value and the pixel smoothing value to reflect the noise level of the pixel point, thereby improving the estimation accuracy of the noise. Finally, the noise level estimation under different brightness intervals is output, providing high-quality input for subsequent image processing.

[0035] Next, the specific steps of the scheme are described. In the above S102, the pixel attribution value of the current pixel point in the image to be processed is determined, including: obtaining the brightness value corresponding to each original image fused to obtain the image to be processed; obtaining the brightness value of the current pixel point; determining the pixel attribution value of the current pixel point according to the brightness value of the current pixel point and the brightness value corresponding to each original image.

[0036] In this embodiment, the pixel attribution value is determined according to the brightness value corresponding to each original image fused to obtain the image to be processed. As mentioned above, when two or more original images are fused, the RAW image with lower brightness is enlarged and then fused with another image. The enlargement coefficient is usually determined based on the relationship between two or more images, such as exposure ratio or time ratio. By taking the brightness value as the basis for judgment, the pixel attribution value of the current pixel point can be determined, that is, which original image it belongs to.

[0037] Further, the following takes two original images fusion as an example of the image to be processed, how to determine the pixel attribution value of the current pixel point is described. According to the brightness value of the current pixel point and the brightness value corresponding to each original image, the pixel attribution value of the current pixel point is determined, including: if the image to be processed is obtained by fusing the first original image and the second original image, the brightness value of the current pixel point is obtained; the brightness value of the current pixel point is compared with the preset brightness threshold value, in the case that the brightness value of the current pixel point is less than or equal to the preset brightness threshold value, the first attribution value is assigned to the current pixel point; or, in the case that the brightness value of the current pixel point is greater than the preset brightness threshold value, the second attribution value is assigned to the current pixel point; wherein, the first attribution value represents that the current pixel point is derived from the first original image, the second attribution value represents that the current pixel point is derived from the second original image, and the preset brightness threshold value is set according to the brightness value corresponding to the first original image in the fusion and the brightness value corresponding to the second original image in the fusion.

[0038] In the embodiment, in combination with Figure 2 As shown in the figure, the HDR fusion image is fused by two original images, which are image 1 of orange part and image 2 of blue part. Among them, the exposure time of image 1 is longer, and the exposure time of image 2 is shorter, that is, image 1 is a relatively bright image, with the increase of scene brightness, the image signal will increase, but it will reach the cutoff position earlier, that is, in the case that image 1 reaches the corresponding cutoff position, the scene brightness will not change. As shown in picture 1, the sun is photographed, and with the increase of scene brightness, the sun will gradually show white, but after reaching the cutoff position, the white part of the sun will not change with the increase of brightness. Image 2 is a relatively dark image, although it also photographs the sun, but it can carry higher scene brightness compared with image 1, so the cutoff position corresponding to image 2 is later than that of image 1, which can carry more scene brightness. When fusing, image 2 needs to be enlarged by a certain proportion (such as the dashed part of image 2 is enlarged to the solid part), so as to be fused with image 1. The proportion of enlargement can be determined based on the relationship of brightness threshold value or exposure ratio of the two images.

[0039] The above is only an introduction to the fusion process for the purpose of understanding the cut-off position, and does not limit how to obtain the HDR image to be processed. If the fusion manner is changed, a required HDR image can also be obtained. For the obtained HDR image, the brightness value of the current pixel point is first obtained, and then the brightness value is compared with a preset brightness threshold value, so that it can be determined whether the pixel attribution value is the first attribution value or the second attribution value, i.e., which original image the pixel belongs to. The preset brightness threshold value can be determined according to the brightness value corresponding to the first original image during fusion and the brightness value corresponding to the second original image during fusion. The brightness value corresponding to the first original image during fusion and the brightness value corresponding to the second original image during fusion include different signal cut-off positions during fusion of the two original images, such as the signal cut-off position 1 corresponding to the image 1 and the signal cut-off position 2 corresponding to the enlarged image 2 as shown in the image 1. Figure 2 If the preset brightness threshold value is set as the brightness corresponding to the signal cut-off position 1, it can be determined that if the brightness value of the current pixel point of the fused HDR image is greater than the preset brightness threshold value, i.e., belongs to the image 2; otherwise, belongs to the image 1. Of course, the preset brightness threshold value can also be a value slightly greater than the brightness corresponding to the signal cut-off position 1, because the brightness value at the signal cut-off position 1 is not completely clear, for example, in the case that the brightness corresponding to the current pixel value is the same as the brightness corresponding to the signal cut-off position 1, it cannot be completely distinguished whether it comes from the image 1 or the image 2, so the preset brightness threshold value can be slightly greater than the brightness corresponding to the signal cut-off position 1, so as to facilitate the distinction of the image source.

[0040] In a feasible embodiment, the pixel attribution value can be set as S_MASK, the first attribution value is 0, and the second attribution value is 1. In this embodiment, if the brightness value of the current pixel point is less than the brightness value corresponding to the signal cut-off position 1, it means that it comes from the image 1, and S_MASK is assigned as 0; if the brightness value of the current pixel point is greater than the brightness value corresponding to the signal cut-off position 1, it means that it comes from the image 2, and S_MASK is assigned as 1.

[0041] In the case of fusion of multiple original images into a to-be-processed image, the above-mentioned manner can also be used. For example, an HDR image is obtained by fusing three images A, B and C, the brightness before fusion is a, b and c respectively; the signal cut-off position of the image A is X, the signal cut-off position of the image B is Y, and a < X < b < Y < c. Then, if the brightness of a pixel point in the fused HDR image is less than X, it is determined to come from the image A; if the brightness of a pixel point is between X and Y, it is determined to come from the image B.

[0042] Optionally, the pixel smooth value of the current pixel point in the image to be processed is determined by: obtaining pixel values of each pixel point in a first preset region centered on the current pixel point; performing filtering calculation on the pixel values of each pixel point in the first preset region by a high-pass filter operator to obtain a high-frequency component corresponding to the first preset region; comparing the high-frequency component corresponding to the first preset region with a preset component threshold value, and assigning a first smooth value to the current pixel point in a case where the high-frequency component corresponding to the first preset region is less than or equal to the preset component threshold value; or assigning a second smooth value to the current pixel point in a case where the high-frequency component corresponding to the first preset region is greater than the preset component threshold value; wherein the first smooth value represents that a region where the current pixel point is located is a flat region; and the second smooth value represents that the region where the current pixel point is located is a texture region, and the high-pass filter operator includes a Sobel operator or a Laplace operator.

[0043] In the embodiment, the pixel smooth value is used to determine whether a region set for each pixel point of the HDR fusion image is a texture region or a flat region. Specifically, whether a region is a texture region can be determined by high-pass filtering of an image and by the size of a high-frequency component of a pixel point. Common high-pass filter operators include a Sobel operator and a Laplace operator. Here, the first preset region refers to a region or window of a preset size centered on a current pixel point, for example, a 3*3 window centered on the current pixel point, as the preset region corresponding to the current pixel point. The preset component threshold value can be set according to actual conditions. In a feasible calculation manner, first, the high-frequency components of each pixel point in the region are calculated, and then the average of the difference between each component and the preset component threshold value is calculated, and a first smooth value or a second smooth value is assigned according to the average size. The first smooth value indicates a flat region, and the second smooth value indicates a texture region.

[0044] For example, the high-frequency components of nine pixel points in a 3*3 window centered on a certain pixel point are calculated, and then the average of the difference between the nine high-frequency components and a preset component threshold value is calculated. Then, a threshold value is set, which can be set according to actual experience. If the threshold value is greater than the threshold value, it is determined that the region is a texture region. Alternatively, a model analysis method is used. The model analyzes historical data to reflect the flat or texture characteristics of the region. The first smooth value can be represented as D_MASK=0, and the second smooth value can be represented as D_MASK=1. It should be noted that although it is determined whether a region is flat or textured, the flat or textured characteristics of the region are reflected in the pixel smooth value of the center point. That is, in the above embodiment, although the nine pixel points finally calculate the pixel smooth value D_MASK corresponding to the center pixel point, each pixel point is the center point of its first preset region, and each of the nine pixel points has a corresponding pixel smooth value D_MASK.

[0045] Optionally, according to the pixel attribution value and the pixel smoothing value of the current pixel point, a local variance value of the current pixel point is calculated under a preset condition, including: taking the current pixel point as the center, filtering out a first pixel point set with a second smoothing value in a second preset region; determining the pixel attribution value of the current pixel point, filtering out pixel points different from the pixel attribution value of the current pixel point in the first pixel point set to obtain a second pixel point set; and calculating the local variance value of the current pixel point according to the current pixel point and the second pixel point set.

[0046] In the embodiment, the setting of the preset condition is described. First, the second preset region is also a region or window with a preset range centered on the current pixel point, which can be the same as the first preset region. Similarly, taking the current pixel point as the center, 9 pixel points in a 3*3 window are calculated. As mentioned above, the 9 pixel points all have corresponding D_MASK. First, the pixels with D_MASK=0 in the second preset region are determined, that is, the pixel points with D_MASK=1 are filtered out to obtain the first pixel point set. For example, in the original 9 pixel points, there are 2 pixel points with D_MASK=1, and the first pixel point set is the remaining 7 pixel points with D_MASK=0. Then, the pixel points different from the pixel attribution value of the current pixel point are filtered out in the first pixel point set to obtain the second pixel point set. That is, in the 7 pixel points with D_MASK=0, if the pixel attribution value S_MASK of the center point is 0, only the pixel points with S_MASK=0 in the 7 points are considered. For example, there are 3 pixel points with S_MASK=1 in the 7 pixel points, and the second pixel point set is the remaining 4 pixel points with S_MASK=0. Then, the local variance value is calculated according to the current pixel point and the remaining 4 pixel points.

[0047] Specifically, the local variance value of the current pixel point is calculated according to the current pixel point and the second pixel point set, and the following formula is used for calculation: (Formula One) Wherein, is the local variance value of the current pixel point, is the pixel value corresponding to the i-th pixel point in the second preset region; is the attribution judgment, in the case that the pixel attribution value of the i-th pixel point is the same as that of the current pixel point, is 1, otherwise 0; is the smoothing judgment, in the case that the pixel smoothing value of the current pixel point is the first smoothing value, is 1, otherwise 0.

[0048] In the embodiment, the purpose of D_MASK=0 is to exclude the influence of the texture region in the second preset region on the noise estimation; taking the same point as the current pixel point of S_MASK is to determine the same image source, so as to avoid the noise error introduced by other images. Through the setting of the preset condition, the estimation accuracy of the noise is improved.

[0049] Optionally, based on the brightness value of each pixel point in the to-be-processed image, the local variance value corresponding to each pixel point is segmented and counted to obtain the noise level estimation of the HDR image in different brightness intervals, including: dividing the brightness of the to-be-processed image into N continuous brightness intervals, where N is an integer greater than 1; according to the brightness value of each pixel point in the to-be-processed image, each pixel point and the corresponding local variance value are divided into the corresponding brightness interval; for each brightness interval, the statistical average value of the local variance values corresponding to all pixel points falling into the interval is calculated; the statistical average value corresponding to each brightness interval is taken as the noise level estimation value of the HDR image in the brightness interval.

[0050] In the embodiment, the noise level estimation of the HDR image in different brightness intervals is obtained according to the brightness, which facilitates the application and processing of other noise reduction algorithms. As shown in Figure 3 , a schematic diagram of a noise level estimation curve is shown. The vertical axis is the noise size, that is, the reflection value of the local variance value; the horizontal axis is the image signal intensity, that is, the size of the image brightness signal; the orange curve represents the noise level of image 1, and the blue curve represents the noise level of image 2. From Figure 3 It can be seen that the brightness of the to-be-processed image is divided into N continuous brightness intervals, and then the local variance value is quantized, so that the noise level estimation value of the HDR image in the brightness interval is obtained, thereby facilitating subsequent noise reduction processing.

[0051] In combination with the image processing method mentioned in any of the above embodiments, the embodiment further provides a system architecture of a specific operation module of image processing, which is composed of an S_MASK generation module, a D_MASK generation module, a local variance calculation module and a statistical module.

[0052] In combination with Figure 4As shown, the process of performing the image processing method on each module is described. First, an HDR image fused is input. Then, the S_MASK generation module generates the S_MASK of each pixel point of the input HDR image, that is, the pixel attribution value. The D_MASK generation module generates the D_MASK pixel smoothing value of each pixel point of the input HDR image. Then, the local variance calculation module calculates the local variance value of each pixel point according to the preset condition. Finally, the statistical module performs pixel brightness statistics and noise evaluation. For specific details, please refer to the image processing method mentioned in the above embodiment, which will not be described here. In this embodiment, the pixel attribution value of each pixel point of the HDR image is calculated to determine the pixel source, the pixel smoothing value is calculated to avoid the interference of the texture region, and then the local variance value is calculated based on the pixel attribution value and the pixel smoothing value to reflect the noise level of the pixel point, thereby improving the estimation accuracy of the noise. Finally, the noise level estimation under different brightness intervals is output, providing high-quality input for subsequent image processing.

[0053] Further, as Figures 1 to 4 As shown in the specific implementation of the method, the embodiment provides an image processing device, which comprises: Figure 5 As shown, the device comprises an acquisition unit 501, a first calculation unit 502, a second calculation unit 503, a processing unit 504, and a statistical unit 505.

[0054] The acquisition unit 501 is configured to acquire a to-be-processed image, wherein the to-be-processed image is an HDR image fused from at least two original images; The first calculation unit 502 is configured to determine a pixel attribution value of a current pixel point in the to-be-processed image, wherein the pixel attribution value is used to determine the original image to which the current pixel point belongs; The second calculation unit 503 is configured to determine a pixel smoothing value of the current pixel point in the to-be-processed image, wherein the pixel smoothing value is used to determine the image smoothing degree of the region where the current pixel point is located; The processing unit 504 is configured to calculate a local variance value of the current pixel point according to the pixel attribution value and the pixel smoothing value of the current pixel point and according to a preset condition; The statistical unit 505 is configured to perform segmented statistics on the local variance value corresponding to each pixel point based on the brightness value of each pixel point in the to-be-processed image, to obtain the noise level estimation of the HDR image under different brightness intervals.

[0055] In a specific application scenario, the first calculation unit 502 is specifically further configured to obtain a brightness value corresponding to each original image of the to-be-processed image fused; obtain the brightness value of the current pixel point; and determine the pixel attribution value of the current pixel point according to the brightness value of the current pixel point and the brightness value corresponding to each original image.

[0056] In a specific application scenario, the first calculation unit 502 is specifically further configured to, if the to-be-processed image is fused from a first original image and a second original image, obtain the brightness value of the current pixel point; compare the brightness value of the current pixel point with a preset brightness threshold value; in a case where the brightness value of the current pixel point is less than or equal to the preset brightness threshold value, assign a first attribution value to the current pixel point; or, in a case where the brightness value of the current pixel point is greater than the preset brightness threshold value, assign a second attribution value to the current pixel point; wherein the first attribution value represents that the current pixel point originates from the first original image, the second attribution value represents that the current pixel point originates from the second original image, and the preset brightness threshold value is set according to a brightness value corresponding to the first original image in the fusion and a brightness value corresponding to the second original image in the fusion.

[0057] In a specific application scenario, the second calculation unit 503 is specifically further configured to obtain pixel values of each pixel point in a first preset region with the current pixel point as the center; perform filtering calculation on the pixel values of each pixel point in the first preset region through a high-pass filtering operator to obtain a high-frequency component corresponding to the first preset region; compare the high-frequency component corresponding to the first preset region with a preset component threshold value; in a case where the high-frequency component corresponding to the first preset region is less than or equal to the preset component threshold value, assign a first smoothing value to the current pixel point; or, in a case where the high-frequency component corresponding to the first preset region is greater than the preset component threshold value, assign a second smoothing value to the current pixel point; wherein the first smoothing value represents that a region in which the current pixel point is located is a flat region; the second smoothing value represents that the region in which the current pixel point is located is a texture region, and the high-pass filtering operator includes a Sobel operator or a Laplace operator.

[0058] In a specific application scenario, the processing unit 504 is specifically further configured to filter out a first pixel point set in which the pixel smoothing value is the second smoothing value in a second preset region with the current pixel point as the center; determine the pixel attribution value of the current pixel point by filtering out pixel points different from the pixel attribution value of the current pixel point in the first pixel point set to obtain a second pixel point set; and calculate a local variance value of the current pixel point according to the current pixel point and the second pixel point set.

[0059] In a specific application scenario, the processing unit 504 is further configured to calculate according to the following formula: wherein, is a local variance value of the current pixel point, is a pixel value corresponding to the i-th pixel point in the second preset region; is a belonging judgment, in the case that the pixel belonging values of the i-th pixel point and the current pixel point are the same, is 1, otherwise 0; is a smoothing judgment, in the case that the pixel smoothing value of the current pixel point is the first smoothing value, is 1, otherwise 0.

[0060] In a specific application scenario, the statistical unit 505 is further configured to divide the brightness of the image to be processed into N continuous brightness intervals, wherein N is an integer greater than 1; according to the brightness value of each pixel point in the image to be processed, each pixel point and the corresponding local variance value are divided into the corresponding brightness interval; for each brightness interval, calculate the statistical average value of the local variance value corresponding to all pixel points falling into the interval; the statistical average value corresponding to each brightness interval is taken as the noise level estimation value of the HDR image in the brightness interval.

[0061] It should be noted that other corresponding descriptions of the functions of the image processing device provided in the embodiment can be referred to the corresponding descriptions in the Figures 1 to 4 , which will not be repeated here.

[0062] Based on the above method as shown in Figures 1 to 4 , accordingly, the embodiment also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the above method as shown in Figures 1 to 4 .

[0063] Based on such understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.), including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the method of each embodiment scenario of the present application.

[0064] Based on the above method as shown in Figures 1 to 4 , and Figure 5To achieve the above object, the virtual device embodiment shown provides an electronic device which can be configured at a computer terminal side or the like, and the device comprises a storage medium and a processor; the storage medium is configured to store a computer program; the processor is configured to execute the computer program to implement the above-mentioned method. Figures 1 to 4 The method shown.

[0065] Optionally, the above-mentioned entity device can further comprise a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, and the like. The user interface can comprise a display, an input unit such as a keyboard, and the like. Optionally, the user interface can further comprise a USB interface, a card reader interface, and the like. The network interface can optionally comprise a standard wired interface, a wireless interface (such as a WI-FI interface), and the like.

[0066] Those skilled in the art can understand that the above-mentioned entity device structure provided by the embodiment does not constitute a limitation on the entity device, and can comprise more or fewer components, or combine certain components, or different component arrangements.

[0067] The storage medium can further comprise an operating system and a network communication module. The operating system is a program for managing hardware and software resources of the above-mentioned entity device, and supports the running of an information processing program and other software and / or programs. The network communication module is configured to realize communication between components in the storage medium, and communication with other hardware and software in the information processing entity device.

[0068] Through the above description of the embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software and necessary general hardware platforms, or by hardware. By applying the scheme of the embodiment, compared with the related art, the pixel attribution value of each pixel point of the HDR image is calculated to determine the pixel source, the pixel smoothing value is calculated to avoid the interference of the texture region, and then the local variance value is calculated based on the pixel attribution value and the pixel smoothing value to reflect the noise level of the pixel point, thereby improving the estimation accuracy of the noise. Finally, the noise level estimation under different brightness intervals is output, providing high-quality input for subsequent image processing.

[0069] It has to be noted that, in the present document, relational terms are intended only to convey a possible relationship between elements or

[0070] The above description is merely that of the specific embodiments of the application and as such is not to be taken in a limiting sense. Various modifications and co nti n uations will be evident to those skilled in the art that do not depart from the spirit and scope of the application as defined by the appended claims. The specific embodiments presented, therefore, are not to be considered in a limiting sense, but are presented for purposes of illustration only in conformance with the above-stated description. It is not the intention to limit the application to the described embodiments but rather the intention is to cover all modifications and alternatives coming within the spirit and scope of the claims.

Claims

1. An image processing method, characterized by, The method comprises: obtaining a to-be-processed image, the to-be-processed image being an HDR image obtained by fusing at least two original images; determining a pixel attribution value of a current pixel point in the to-be-processed image, the pixel attribution value being used to determine an original image to which the current pixel point belongs; determining a pixel smoothing value of the current pixel point in the to-be-processed image, the pixel smoothing value being used to determine a smoothness of an image in a region where the current pixel point is located; calculating a local variance value of the current pixel point according to the pixel attribution value and the pixel smoothing value of the current pixel point and under a preset condition; based on luminance values of each pixel point in the to-be-processed image, performing segmented statistics on the local variance value corresponding to each pixel point to obtain noise level estimation of the HDR image under different luminance intervals.

2. The method of claim 1, wherein, The method comprises: obtaining a luminance value corresponding to each original image fused to obtain the to-be-processed image; obtaining a luminance value of the current pixel point; determining the pixel attribution value of the current pixel point according to the luminance value of the current pixel point and the luminance value corresponding to each original image.

3. The method of claim 2, wherein, The method comprises: if the to-be-processed image is obtained by fusing a first original image and a second original image, obtaining the luminance value of the current pixel point; comparing the luminance value of the current pixel point with a preset luminance threshold value, and in a case where the luminance value of the current pixel point is less than or equal to the preset luminance threshold value, assigning a first attribution value to the current pixel point; or, in a case where the luminance value of the current pixel point is greater than the preset luminance threshold value, assigning a second attribution value to the current pixel point; wherein the first attribution value represents that the current pixel point is derived from the first original image, the second attribution value represents that the current pixel point is derived from the second original image, and the preset luminance threshold value is set according to a luminance value corresponding to the first original image in the fusion and a luminance value corresponding to the second original image in the fusion.

4. The method of claim 3, wherein, The method comprises: obtaining pixel values of each pixel point in a first preset region with the current pixel point as the center; performing filter calculation on the pixel values of each pixel point in the first preset region by a high-pass filter operator to obtain a high-frequency component corresponding to the first preset region; comparing the high-frequency component corresponding to the first preset region with a preset component threshold value, and in a case where the high-frequency component corresponding to the first preset region is less than or equal to the preset component threshold value, assigning a first smoothing value to the current pixel point; or, in a case where the high-frequency component corresponding to the first preset region is greater than the preset component threshold value, assigning a second smoothing value to the current pixel point; wherein the first smoothing value represents that a region where the current pixel point is located is a flat region, and the second smoothing value represents that the region where the current pixel point is located is a texture region, and the high-pass filter operator comprises a Sobel operator or a Laplace operator.

5. The method of claim 4, wherein, The local variance value of the current pixel point is calculated according to the pixel attribution value and the pixel smoothing value of the current pixel point under a preset condition, including: Filtering out a first pixel point set with the pixel smoothing value being the second smoothing value in a second preset region centered on the current pixel point; Determining the pixel attribution value of the current pixel point, filtering out pixel points different from the pixel attribution value of the current pixel point in the first pixel point set to obtain a second pixel point set; Calculating the local variance value of the current pixel point according to the current pixel point and the second pixel point set.

6. The method of claim 5, wherein, The local variance value of the current pixel point is calculated according to the current pixel point and the second pixel point set, and the following formula is used for calculation: ； wherein, is a local variance value of the current pixel point, is a pixel value corresponding to the i-th pixel point in the second preset region; is a belonging judgment, in a case where the i-th pixel point and the current pixel point have the same pixel belonging value, is 1, otherwise 0; is a smoothing judgment, in a case where the pixel smoothing value of the current pixel point is the first smoothing value, is 1, otherwise 0.

7. The method of claim 2, wherein, The local variance value corresponding to each pixel point is segmented and counted based on the brightness value of each pixel point in the to-be-processed image, and noise level estimation of the HDR image under different brightness intervals is obtained, including: The brightness of the to-be-processed image is divided into N continuous brightness intervals, where N is an integer greater than 1; According to the brightness value of each pixel point in the to-be-processed image, each pixel point and the corresponding local variance value are divided into the corresponding brightness interval; For each brightness interval, calculate the statistical average value of the local variance value corresponding to all pixel points falling into the interval; The statistical average value corresponding to each brightness interval is taken as the noise level estimation value of the HDR image in the brightness interval.

8. An image processing apparatus characterized by comprising: It includes: An acquisition unit configured to acquire a to-be-processed image, the to-be-processed image being an HDR image obtained by fusing at least two original images; A first calculation unit configured to determine a pixel attribution value of a current pixel point in the to-be-processed image, the pixel attribution value being used to determine the original image to which the current pixel point belongs; A second calculation unit configured to determine a pixel smoothing value of the current pixel point in the to-be-processed image, the pixel smoothing value being used to determine the image smoothing degree of the region where the current pixel point is located; A processing unit configured to calculate a local variance value of the current pixel point according to the pixel attribution value and the pixel smoothing value of the current pixel point under a preset condition; A statistical unit configured to segment and count the local variance value corresponding to each pixel point based on the brightness value of each pixel point in the to-be-processed image, and obtain noise level estimation of the HDR image under different brightness intervals.

9. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, The processor executes the computer program to implement the method in any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method in any one of claims 1 to 7.

Images