Image denoising method, electronic device, and storage medium

Abstract

Embodiments of the present application disclose an image denoising method, an electronic device and a storage medium. The method comprises: determining a flat region and a non-flat region in a first image to be processed according to pixel values of at least one color channel of each pixel point in the first image; performing color denoising processing on the flat region according to a first denoising intensity to obtain a first denoising result; performing color denoising processing on the non-flat region according to a second denoising intensity to obtain a second denoising result, wherein the first denoising intensity is greater than the second denoising intensity; obtaining edge information of the first image; and fusing the first image, the first denoising result and the second denoising result based on the edge information to obtain a target image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an image denoising method, an electronic device and a storage medium. BACKGROUND

[0002] The noise distributed in the low frequency band of the image is generally called color noise, which has low noise amplitude and large scale. However, the general denoising algorithm is usually designed for noise with high frequency, and it is difficult to remove the low frequency color noise in the image well. In the related art, for the color noise in the image, the image is converted into the YUV color space, the Y channel is used as a guide image, and the U channel and the V channel are guided filtered respectively to remove the color noise in the image. This method is prone to color overflow in the area with different brightness and colors, resulting in poor image denoising effect. SUMMARY

[0003] Embodiments of the present application provide an image denoising method, an electronic device and a storage medium to solve the technical problem of poor image denoising effect in the related art.

[0004] According to a first aspect of the present application, an image denoising method is disclosed, the method comprising:

[0005] Determining a flat area and a non-flat area in the first image according to the pixel value of at least one color channel of each pixel point in the first image to be processed;

[0006] Performing color noise reduction processing on the flat area according to a first denoising intensity to obtain a first denoising result, and performing color noise reduction processing on the non-flat area according to a second denoising intensity to obtain a second denoising result, wherein the first denoising intensity is greater than the second denoising intensity;

[0007] Obtaining edge information of the first image;

[0008] Fusing the first image, the first denoising result and the second denoising result based on the edge information to obtain a target image.

[0009] According to a second aspect of the present application, an electronic device is disclosed, comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the image denoising method as in the first aspect.

[0010] According to a third aspect of the present application, a computer readable storage medium is disclosed, which stores a computer program / instruction, and the computer program / instruction is executed by a processor to implement the image denoising method as in the first aspect.

[0011] According to a fourth aspect of the present application, a computer program product is disclosed, comprising computer programs / instructions which, when executed by a processor, implement the image denoising method as in the first aspect.

[0012] In the embodiments of the present application, the flat region and the non-flat region in the first image are determined according to the pixel value of at least one color channel of each pixel point in the first image to be processed; the flat region is subjected to color noise reduction processing according to a first denoising intensity to obtain a first denoising result; and the non-flat region is subjected to color noise reduction processing according to a second denoising intensity to obtain a second denoising result, wherein the first denoising intensity is greater than the second denoising intensity; the edge information of the first image is obtained; and the first image, the first denoising result and the second denoising result are fused based on the edge information to obtain a target image.

[0013] It can be seen that, in the embodiments of the present application, since the pixel change of the flat region in the image is mainly caused by noise, the denoising intensity can be increased when the flat region is subjected to denoising, so that the color noise in the flat region can be removed more cleanly; and since the pixel change of the non-flat region in the image is mainly caused by texture or edge, and noise only accounts for a small part, the denoising intensity can be reduced when the non-flat region is subjected to denoising, so that the color noise in the non-flat region can be removed more cleanly while the texture and edge features of the image are maintained, avoiding the disappearance or distortion of the texture and edge of the image; and since color overflow usually occurs at the junction position of regions with the same brightness and different colors, and the edge information of the image can reflect the aforementioned junction position, the original image and its denoising result can be fused based on the edge information of the image, so that the color noise in the image can be removed more cleanly while the problem of obvious color overflow is avoided, improving the image denoising effect. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 is an example diagram of a neighborhood of a pixel point provided by the embodiments of the present application;

[0015] Figure 2 is a flowchart of an image denoising method provided by the embodiments of the present application;

[0016] Figure 3 is a flowchart of an implementation of step 201 provided by the embodiments of the present application;

[0017] Figure 4 is a flowchart of an implementation of step 202 provided by the embodiments of the present application;

[0018] Figure 5 is a flowchart of an implementation of step 204 provided by the embodiments of the present application;

[0019] Figure 6is a structural schematic diagram of an image noise reduction device provided by an embodiment of the present application.

[0020] Figure 7 is a structural block diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0022] It should be noted that, for the method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the action order described, because according to the present application, certain steps can be performed in other order or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification all belong to preferred embodiments, and the actions involved are not necessarily the necessary of the present application.

[0023] In recent years, important progress has been made in the research of computer vision, deep learning, machine learning, image processing, image recognition and other technologies based on artificial intelligence. Artificial intelligence (AI) is a new science and technology that studies and develops theories, methods, technologies and application systems for simulating and extending human intelligence. Artificial intelligence is a comprehensive discipline involving chips, big data, cloud computing, the Internet of Things, distributed storage, deep learning, machine learning, neural networks and many other technology categories. Computer vision, as an important branch of artificial intelligence, is specifically to let the machine recognize the world. Computer vision technology usually includes face recognition, live detection, fingerprint recognition and anti-fraud verification, biometric recognition, face detection, pedestrian detection, target detection, pedestrian recognition, image processing, image recognition, image semantic understanding, image retrieval, character recognition, video processing, video content recognition, behavior recognition, three-dimensional reconstruction, virtual reality, augmented reality, simultaneous localization and mapping, computational photography, robot navigation and positioning, and other technologies. With the research and progress of artificial intelligence technology, this technology has been applied in many fields, such as security, city management, traffic management, building management, park management, face pass, face attendance, logistics management, warehouse management, robots, intelligent marketing, computational photography, mobile imaging, cloud services, smart home, wearable devices, unmanned vehicles, autonomous driving, intelligent medical care, face payment, face unlocking, fingerprint unlocking, face and certificate verification, smart screen, smart TV, camera, mobile Internet, network live broadcast, beauty, makeup, medical cosmetology, intelligent temperature measurement, and other fields.

[0024] This application provides an image noise reduction method, an electronic device, and a storage medium. For ease of understanding, the application scenarios and some related concepts of this application embodiment are first introduced below.

[0025] Color noise: Noise distributed in the lower frequency band of an image, with low noise amplitude and large scale.

[0026] The neighborhood of a pixel: a certain range of pixels centered on that pixel, usually a rectangular area, for example... Figure 1 As shown, the size of image 10 is 8×8. For pixel A35, if the radius of the neighborhood is 3, then the neighborhood centered on pixel A35 is image region 12, which is a 5×5 rectangular region.

[0027] It should be noted that for boundary pixels like pixel A11 whose actual number of pixels in the neighborhood is less than 5×5, the missing pixels in its neighborhood can be filled by padding, or no processing can be performed.

[0028] The following section describes an image denoising method provided by an embodiment of this application.

[0029] Figure 2 This is a flowchart of an image denoising method provided in an embodiment of this application, such as... Figure 2 As shown, the method may include the following steps: step 201, step 202, step 203 and step 204;

[0030] In step 201, flat and non-flat regions in the first image are determined based on the pixel values ​​of at least one color channel of each pixel in the first image to be processed.

[0031] In this embodiment of the application, the first image can be an RGB image or a color image of other formats. For ease of understanding and description, the following will mainly use an RGB image as the first image as an example for illustration. The processing of other image formats is similar to that of RGB images, and will not be described in detail here.

[0032] The two-dimensional Fourier transform of an image transforms the spatial representation of the image into a frequency domain representation. Flat regions in an image can be understood as changing relatively slowly and having a low frequency, while the edges and textured regions of the image change more rapidly, i.e., the high-frequency components.

[0033] In the embodiments of the present application, considering that the pixel change of the flat region in the image is mainly caused by noise, the noise reduction strength can be increased; and for the non-flat region such as texture or edge in the image, the pixel change is mainly caused by texture or edge, and noise only accounts for a small part, so the noise reduction strength needs to be controlled to keep the texture and edge features of the image as much as possible. Therefore, for the first image needing noise reduction, the flat region and the non-flat region in the first image can be determined first, then the flat region is processed by the large-strength noise reduction, and the non-flat region is processed by the small-strength noise reduction.

[0034] In the embodiments of the present application, considering that the color channel value change of the pixel point in the image can reflect the change of the pixel point in the image, and the change of the pixel point in the flat region and the non-flat region in the image is different, the flat region and the non-flat region in the first image can be determined according to the pixel value of at least one color channel of each pixel point in the first image.

[0035] Taking the first image as an RGB image as an example, the flat region and the non-flat region in the RGB image can be determined according to the pixel value of at least one color channel of the R channel, the G channel and the B channel of each pixel point in the RGB image.

[0036] In step 202, the flat region is processed by the first noise reduction strength to obtain a first noise reduction result, and the non-flat region is processed by the second noise reduction strength to obtain a second noise reduction result, wherein the first noise reduction strength is greater than the second noise reduction strength.

[0037] In the embodiments of the present application, considering that the pixel change of the flat region in the image is mainly caused by noise, the noise reduction strength can be increased; and for the non-flat region such as texture or edge in the image, the pixel change is mainly caused by texture or edge, and noise only accounts for a small part, so the noise reduction strength needs to be controlled to keep the texture and edge features of the image as much as possible.

[0038] In the embodiments of the present application, the first noise reduction result is the noise reduction result of each pixel point in the flat region of the first image, and the second noise reduction result is the noise reduction result of each pixel point in the non-flat region of the first image.

[0039] In step 203, the edge information of the first image is obtained.

[0040] In the embodiments of the present application, the edge information can include at least one of the following: information of strong edges and information of weak edges in the image.

[0041] In the embodiments of the present application, in order to avoid the problem of obvious color overflow in the target image after noise reduction, edge information of the first image can be obtained, and the first image and the noise reduction result thereof are fused based on the edge information to obtain the target image.

[0042] In step 204, the first image, the first noise reduction result and the second noise reduction result are fused based on the edge information to obtain the target image.

[0043] In the embodiments of the present application, the weight value of the first image in the fusion, the weight value of the first noise reduction result in the fusion and the weight value of the second noise reduction result in the fusion can be obtained based on the edge information, and then the first image, the first noise reduction result and the second noise reduction result are fused based on the weight values to obtain the target image.

[0044] As can be seen from the above embodiments, in the embodiments, the flat region and the non-flat region in the first image are determined according to the pixel value of at least one color channel of each pixel point in the first image to be processed; the flat region is subjected to color noise reduction processing according to the first noise reduction strength to obtain the first noise reduction result; and the non-flat region is subjected to color noise reduction processing according to the second noise reduction strength to obtain the second noise reduction result, wherein the first noise reduction strength is greater than the second noise reduction strength; the edge information of the first image is obtained; and the first image, the first noise reduction result and the second noise reduction result are fused based on the edge information to obtain the target image.

[0045] As can be seen, in the embodiments of the present application, since the pixel change in the flat region of the image is mainly caused by noise, the noise reduction strength can be increased when the flat region is subjected to noise reduction, so that the color noise in the flat region can be removed more cleanly; and since the pixel change in the non-flat region of the image is mainly caused by texture or edge, and noise only accounts for a small part, the noise reduction strength can be reduced when the non-flat region is subjected to noise reduction, so that the color noise in the non-flat region can be removed more cleanly while the texture and edge features of the image are maintained, avoiding the disappearance or distortion of the texture and edge of the image; and since color overflow usually occurs at the junction position of regions with the same brightness and different colors, and the edge information of the image can reflect the aforementioned junction position, the image and the noise reduction result thereof can be fused based on the edge information of the image, so that the color noise in the image can be removed more cleanly while avoiding the problem of obvious color overflow, improving the image noise reduction effect.

[0046] Compared with the related art, considering that the pixel change of a flat region in an image is mainly caused by noise, the noise reduction strength can be increased; and considering that the pixel change of a texture or edge region in the image is mainly caused by the texture or edge, noise only accounts for a small part, and the noise reduction strength needs to be controlled to keep the texture and edge features of the image as much as possible. In the embodiment of the present application, for a first image that needs to be denoised, the type of each pixel point in the first image can be determined first, and the noise reduction strength of each pixel point can be determined according to the type of each pixel point. For each flat pixel point in the first image, the flat pixel point is denoised with a large strength according to the pixel values of the pixel points in a large neighborhood in which the flat pixel point is located. For each non-flat pixel point, the neighborhood radius of the non-flat pixel point is reduced, and the non-flat pixel point is denoised with a small strength according to the pixel values of the pixel points in a small neighborhood in which the non-flat pixel point is located. In this way, while reducing the color noise in the image, the texture and edge of the image are prevented from disappearing or being distorted, and the image denoising effect is improved.

[0047] In another embodiment provided in the present application, as shown in Figure 3 the step 201 can include the following steps: step 301 and step 302.

[0048] In the step 301, for each pixel point in the first image to be processed, the variance value of a first neighborhood centered at the pixel point in each color channel is calculated, where the radius of the first neighborhood is greater than 1.

[0049] In the embodiment of the present application, the radius of the first neighborhood can be set by debugging.

[0050] In the embodiment of the present application, considering that the variance value of an image region in each color channel can reflect the change of the pixel points in the image region, for each pixel point in the first image, the variance of the first neighborhood centered at the pixel point in each color channel can be calculated, and the local contrast can be represented by the variance.

[0051] In an example, taking the first image as an RGB image as an example, for each pixel point in the first image, taking the pixel point A35 in Figure 1 as an example, the variance VarR of the first neighborhood centered at the pixel point A35 in the R channel, the variance VarG of the first neighborhood centered at the pixel point A35 in the G channel, and the variance VarB of the first neighborhood centered at the pixel point A35 in the B channel are calculated.

[0052] In the step 302, the maximum variance value is selected from the variance values in each color channel. If the maximum variance value is greater than a first threshold value, it is determined that the region in which the pixel point is located is a non-flat region. If the maximum variance value is less than or equal to the first threshold value, it is determined that the region in which the pixel point is located is a flat region.

[0053] In the embodiments of the present application, considering that the variances of the first neighborhood centered at a pixel point in different color channels are usually different, in order to ensure the accuracy of the type determination result, the maximum variance in different color channels can be used as the criterion.

[0054] In the embodiments of the present application, the first threshold value can be set in advance, and the first threshold value is the threshold value of the local contrast. In actual application, the value of the first threshold value can be set through debugging.

[0055] In the embodiments of the present application, for a pixel point, if the maximum variance corresponding to the pixel point is not greater than the first threshold value, it is considered that the first neighborhood in which the pixel point is currently located does not contain other pixel points with different chroma from the pixel point, and it is determined that the pixel point belongs to the flat region; if the maximum variance corresponding to the pixel point is greater than the first threshold value, it is considered that the first neighborhood in which the pixel point is currently located contains other pixel points with different chroma from the pixel point, and it is determined that the pixel point belongs to the non-flat region.

[0056] In the example in the above step, the maximum variance max{VarR, VarG, VarB} is selected from the variance VarR of the first neighborhood in the R channel, the variance VarG of the first neighborhood in the G channel and the variance VarB of the first neighborhood in the B channel, and it is determined whether max{VarR, VarG, VarB} is greater than the first threshold value. If yes, it is determined that the pixel point A35 belongs to the non-flat region, otherwise, it is determined that the pixel point A35 belongs to the flat region.

[0057] In the embodiments of the present application, in order to improve the efficiency, starting from the first pixel point in the first image, it is determined whether the first pixel point belongs to the non-flat region or the flat region according to the color channel values of the first neighborhood centered at the first pixel point, then it is determined whether the second pixel point belongs to the non-flat region or the flat region according to the color channel values of the first neighborhood centered at the second pixel point, and the iteration is performed to complete the traversal of all pixel points in the first image, and the region determination result of all pixel points is obtained.

[0058] It can be seen that in the embodiments of the present application, the variances of the pixel point in all color channels can be calculated in combination with the pixel values of the pixel point in all color channels, the maximum variance is used as the judgment criterion to determine whether the pixel point belongs to the non-flat region or the flat region, and the accuracy of the region determination result of each pixel point in the first image can be ensured.

[0059] In another embodiment provided by the present application, as shown in Figure 4 The step 202 can include the following steps: step 401, step 402 and step 403.

[0060] In step 401, the first image is converted to the YUV color space to obtain the original U channel value and the original V channel value corresponding to each pixel point of the first image.

[0061] In the embodiments of the present application, considering the U channel and the V channel in the YUV color space, chroma (Chrominance or Chroma) is used to describe the color and saturation of an image, and the U channel and the V channel are related to the image color noise intensity. Therefore, when the first image is subjected to the color noise reduction processing, the first image can be converted to the YUV color space, and the U channel and the V channel in the YUV color space are processed.

[0062] In step 402, for each pixel point in the flat area of the first image, the original U channel value of the second neighborhood centered on the pixel point is used to calculate the noise reduction U channel value corresponding to the pixel point, and the original V channel value of the second neighborhood centered on the pixel point is used to calculate the noise reduction V channel value corresponding to the pixel point.

[0063] In the embodiments of the present application, the radius of the second neighborhood and the radius of the third neighborhood are both the spatial domain radius of the color noise reduction, which is used to represent the intensity of the color noise reduction. The greater the spatial domain radius of the color noise reduction, the greater the corresponding intensity of the color noise reduction, and the smaller the spatial domain radius of the color noise reduction, the smaller the corresponding intensity of the color noise reduction.

[0064] In the embodiments of the present application, the radius of the second neighborhood can be the same as the radius of the first neighborhood, or can be different, but the radius of the second neighborhood and the radius of the first neighborhood are both greater than the radius of the third neighborhood.

[0065] In the embodiments of the present application, for each pixel point in the flat area of the first image, in order to reduce the color noise of each pixel point cleanly, the original U channel value and the original V channel value of each pixel point in the larger neighborhood (i.e., the second neighborhood) in which the pixel point is located can be used to perform a larger intensity of noise reduction processing on the pixel point.

[0066] In one example, the radius of the second neighborhood is 3, and the radius of the third neighborhood is 1. Figure 1 Taking the pixel point A35 in the image in FIG. 3 as an example, if the pixel point A35 is a pixel point in the flat area, the original U channel values of the pixel points A13, A14, A15, A16, A17, A23, A24, A25, A26, A27, A33, A34, A35, A36, A37, A43, A44, A45, A46, A47, A53, A54, A55, A56 and A57 in the second neighborhood centered on the pixel point A35 are used to calculate the noise reduction U channel value of the pixel point A35, and the original V channel values of the pixel points are used to calculate the noise reduction V channel value of the pixel point A35.

[0067] In this embodiment of the application, in order to improve noise reduction efficiency, for each pixel in the flat area, color noise reduction processing can be performed pixel by pixel, starting from the first pixel and proceeding from left to right and from top to bottom.

[0068] In some embodiments, step 402 above may include the following steps:

[0069] For each pixel in the flat region of the first image, calculate the median or mean of the original U channel values ​​of the second neighborhood centered on the pixel, and determine the result as the denoised U channel value corresponding to the pixel; calculate the median or mean of the original V channel values ​​of the second neighborhood centered on the pixel, and determine the result as the denoised V channel value corresponding to the pixel.

[0070] In one example, the radius of the second neighborhood is 3, still using Figure 1 Taking pixel A35 as an example, the median or mean of the original U-channel values ​​of pixels A13, A14, A15, A16, A17, A23, A24, A25, A26, A27, A33, A34, A35, A36, A37, A43, A44, A45, A46, A47, A53, A54, A55, A56, and A57 in the second neighborhood centered on pixel A35 is calculated. The median or mean obtained above is determined as the noise reduction U-channel value of pixel A35, thereby completing the U-channel color noise reduction of pixel A35.

[0071] Similarly, the median or mean of the original V channel values ​​of pixels A13, A14, A15, A16, A17, A23, A24, A25, A26, A27, A33, A34, A35, A36, A37, A43, A44, A45, A46, A47, A53, A54, A55, A56, and A57 in the second neighborhood centered on pixel A35 is calculated, and the median or mean obtained above is determined as the noise-reduced V channel value of pixel A35, thereby completing the V channel color noise reduction of pixel A35.

[0072] As can be seen, in this embodiment, for each pixel in a flat region, the median or mean of the original U-channel values ​​and the median or mean of the original V-channel values ​​of the second neighborhood centered on that pixel are calculated, and the calculation results are used as the noise reduction result for that pixel. Since the aforementioned median or mean can effectively correct and balance the colors between pixels, the noise reduction effect on each pixel in the flat region is relatively good.

[0073] In step 403, for each pixel in the non-flat region of the first image, the denoising U-channel value corresponding to the pixel is calculated based on the original U-channel value of the third neighborhood centered on the pixel; the denoising V-channel value corresponding to the pixel is calculated based on the original V-channel value of the third neighborhood centered on the pixel; wherein the radius of the second neighborhood and the radius of the third neighborhood are both greater than 1, and the radius of the second neighborhood is greater than the radius of the third neighborhood.

[0074] In this embodiment, for each pixel in the non-flat region of the first image, in order to preserve the texture and edge features of the image as much as possible, it is necessary to control the noise reduction intensity. The noise reduction radius can be adaptively reduced, that is, compared with the radius of the second neighborhood of the pixel in the flat region, the radius of the neighborhood of each pixel in the non-flat region, i.e., the radius of the third neighborhood, is reduced. Based on the original U channel value and the original V channel value of the pixel in the smaller neighborhood, the pixel is subjected to a smaller intensity of noise reduction processing.

[0075] In this embodiment, the noise reduction radius can be adaptively reduced based on the center-sourround theory to obtain the radius of the third neighborhood; where center-sourround is the brightness perception theory proposed by Blommaert in his 1990 paper "An object-oriented model for brightness perception".

[0076] In one example, the radius of the third neighborhood is 2. Figure 1 Taking pixel A75 as an example, if pixel A75 is a pixel in a non-flat region, then the noise-reduced U-channel value of pixel A75 is calculated based on the original U-channel values ​​of pixels A64, A65, A66, A74, A75, A76, A84, A85, and A86 in the third neighborhood centered on pixel A75. Similarly, the noise-reduced V-channel value of pixel A75 is calculated based on the original V-channel values ​​of the above pixels.

[0077] In this embodiment of the application, in order to improve noise reduction efficiency, for each pixel in the non-flat area, color noise reduction processing can be performed pixel by pixel, starting from the first pixel and proceeding from left to right and from top to bottom.

[0078] In some embodiments, the radius of the third neighborhood can be determined in the following way:

[0079] For each pixel point in the non-flat region, a target color channel with a maximum variance value of a second neighborhood centered at the pixel point is determined; a target variance value of the target color channel of a candidate neighborhood centered at the pixel point and with a radius of Rs is calculated, where Rs = R - 1, and R is the radius of the second neighborhood.

[0080] If the target variance value is less than or equal to a third threshold value, Rs is determined as the radius of the third neighborhood; if the target variance value is greater than the third threshold value, the radius of the candidate neighborhood is continuously reduced in the same step-by-step manner until the target variance value of the target color channel of the candidate neighborhood is less than the third threshold value or the radius of the candidate neighborhood is 1, and the final reduced radius is determined as the radius of the third neighborhood.

[0081] In the embodiments of the present application, in order to ensure the accuracy of the determination result of the radius of the third neighborhood, the radius can be gradually reduced by a certain adjustment step based on the radius of the second neighborhood until the requirement of the non-flat region for the noise reduction strength is met, and the radius of the third neighborhood is obtained. The neighborhood after each reduction of the radius is referred to as a candidate neighborhood.

[0082] In the embodiments of the present application, the radius of the candidate neighborhood can be reduced by one pixel unit each time, that is, the step is 1. For example, if the radius of the second neighborhood is 10, the radius is reduced from 10 to 9 during the adaptive adjustment of the noise reduction radius. If the condition is not met, the radius is further reduced to 8, and so on, until the variance of the target color channel of the candidate neighborhood is less than the third threshold value or the radius of the candidate neighborhood is 1, and the final reduced radius is determined as the radius of the third neighborhood.

[0083] As can be seen, in the embodiments of the present application, for each pixel point in the non-flat region, the noise reduction radius of the pixel point can be locally and adaptively adjusted to control the noise reduction strength, so as to maintain the texture and edge features of the image, and thus the noise reduction effect of the non-flat region pixel points in the first image is good.

[0084] In some embodiments, the step 403 can include the following steps: step 4031 and step 4032.

[0085] In step 4031, for each pixel point in the non-flat region of the first image, a first difference value between the original U channel value corresponding to the pixel point and the original U channel value of other pixel points in the third neighborhood except the pixel point is calculated, and a second difference value between the original V channel value corresponding to the pixel point and the original V channel value of the other pixel points is calculated.

[0086] In the embodiments of the present application, the first difference value is used to represent the change size of the two pixel points in the U channel, and the second difference value is used to represent the change size of the two pixel points in the V channel, wherein the smaller the difference value is, the smaller the change is, and the larger the difference value is, the larger the change is.

[0087] In one example, if there are 9 pixel points in the third neighborhood, the difference values between the original U channel value of the center pixel point of the third neighborhood and the original U channel values of the other 8 pixel points are calculated respectively to obtain 8 first difference values; similarly, 8 second difference values are calculated.

[0088] In step 4032, for each pixel point in the non-flat region of the first image, the median value of the original U channel value corresponding to the pixel point and the original U channel value of the reference pixel point is calculated, and the calculation result is determined as the denoised U channel value corresponding to the pixel point, and the median value of the original V channel value corresponding to the pixel point and the original V channel value of the reference pixel point is calculated, and the calculation result is determined as the denoised V channel value corresponding to the pixel point; wherein the reference pixel point is the other pixel point in the third neighborhood whose first difference value and second difference value are less than the second threshold value.

[0089] In the embodiments of the present application, the second threshold value can be set in advance, and the second threshold value is a threshold value of the local difference value. In actual application, the value of the second threshold value can be set through debugging.

[0090] In the embodiments of the present application, for the other pixel points except the center pixel point of the third neighborhood, if the first difference value and the second difference value corresponding to the other pixel points are less than the second threshold value, it is considered that the color of the other pixel points is the same as that of the center pixel point, and no color jump occurs, and the other pixel points are used as the reference pixel points to perform the color noise reduction processing on the center pixel point of the third neighborhood.

[0091] In the embodiments of the present application, for each pixel point in the non-flat region, the denoising result of the pixel point is calculated based on the original U channel value and V channel value of the reference pixel point in the third neighborhood centered on the pixel point, which can well correct and balance the color between the pixel points, and thus the denoising effect on each pixel point in the non-flat region is good.

[0092] It can be seen that, in the embodiment of the present application, for the first image requiring noise reduction, the flat region and the non-flat region in the first image can be determined first, for each pixel point in the flat region of the first image, the pixel point is subjected to a large-intensity noise reduction processing according to the original U channel value and the original V channel value corresponding to each pixel point in a large neighborhood where the pixel point is located, and for each pixel point in the non-flat region, the neighborhood radius where the pixel point is located is reduced, the pixel point is subjected to a small-intensity noise reduction processing according to the original U channel value and the original V channel value corresponding to each pixel point in a small neighborhood where the pixel point is located, so that the color noise in the image is reduced, and the texture and the edge of the image are prevented from disappearing or being distorted, and the image noise reduction effect is improved.

[0093] In another embodiment provided by the present application, the step 203 can include the following steps:

[0094] For each pixel point in the first image, the edge intensity values of the pixel point in each color channel are detected, and the maximum edge intensity value is selected; if the maximum edge intensity value is less than the fourth threshold value, the edge information corresponding to the pixel point is set to 0, and if the maximum edge intensity value is greater than or equal to the fourth threshold value, the edge information corresponding to the pixel point is set to the maximum edge intensity value, and the edge information of the first image is obtained.

[0095] In the embodiment of the present application, the fourth threshold value can be set in advance, and the fourth threshold value is a threshold value for edge detection. In actual application, the value of the fourth threshold value can be set through debugging.

[0096] In the embodiment of the present application, the information of the strong edge in the first image can be extracted through the fourth threshold value.

[0097] In one example, the fourth threshold value is thmap, the first image is an RGB image, the sobel can be used to detect the edge intensity of the R channel, the G channel and the B channel of the first image, which are denoted as e1, e2 and e3, the maximum value e of the edge intensity is max{e1, e2, e3}, the edge intensity of the pixel point whose e is less than thmap is set to 0, and the edge map map is obtained.

[0098] In the embodiment of the present application, in order to avoid the layered phenomenon when the fusion is performed based on the edge information, the edge map can be subjected to a smoothing filtering processing to obtain the final edge information.

[0099] In one example, the edge map is subjected to a smoothing filtering processing by using a 3*3 Gaussian filter to obtain edgemap, i.e., the edge information.

[0100] In another embodiment provided by the present application, as shown in Figure 5 the step 204 can include the following steps: step 501, step 502 and step 503.

[0101] In step 501, based on the edge information of the first image, the original U channel value corresponding to each pixel point of the first image and the de-noising U channel value are weighted to obtain the target U channel value corresponding to each pixel point of the first image; and the original V channel value corresponding to each pixel point of the first image and the de-noising V channel value are weighted to obtain the target V channel value corresponding to each pixel point of the first image.

[0102] In one example, for any pixel point in the first image, the initial U channel value corresponding to the pixel point is U, the de-noising U channel value corresponding to the pixel point is U_new, the initial V channel value corresponding to the pixel point is V, the de-noising V channel value corresponding to the pixel point is V_new, and edgemap is the pixel value of the corresponding pixel position in the edge image of the first image, then the target U channel value corresponding to the pixel point is Us=edgemap*U+(1-edgemap)*U_new, and the target V channel value corresponding to the pixel point is Vs=edgemap*V+(1-edgemap)*V_new.

[0103] In step 502, based on the original U channel value and the original V channel value corresponding to each pixel point of the first image, the target U channel value and the target V channel value corresponding to each pixel point of the first image are guided filtering processing to obtain a filtering result.

[0104] In the embodiment of the application, based on the original U channel value and the original V channel value corresponding to each pixel point of the first image, the target U channel value and the target V channel value corresponding to each pixel point of the first image are guided filtering to further correct the de-noising result to obtain the YUV image after color de-noising, that is, the filtering result.

[0105] In step 503, the filtering result is color space conversion to obtain a target image in the same color space as the first image.

[0106] For example, if the first image is an RGB image, the YUV image after color de-noising is converted to the RGB color space to obtain a target image, and the target image is also an RGB image.

[0107] It can be seen that in the embodiment of the application, the U channel value of the original image and the de-noised U channel value can be fused according to the edge information of the first image, and the V channel value of the original image and the de-noised V channel value can be fused, so that the obvious color overflow of the de-noised image at the strong edge position is avoided, and the image de-noising effect is improved.

[0108] Figure 6 is a structural schematic diagram of an image de-noising device provided by the embodiment of the application, as Figure 6As shown, the image denoising apparatus 600 can include a determining module 601, a denoising module 602, an obtaining module 603, and a fusing module 604.

[0109] The determining module 601 is configured to determine a flat region and a non-flat region in the first image according to pixel values of at least one color channel of each pixel point in the first image to be processed.

[0110] The denoising module 602 is configured to perform color noise reduction processing on the flat region according to a first denoising intensity to obtain a first denoising result, and perform color noise reduction processing on the non-flat region according to a second denoising intensity to obtain a second denoising result, wherein the first denoising intensity is greater than the second denoising intensity.

[0111] The obtaining module 603 is configured to obtain edge information of the first image.

[0112] The fusing module 604 is configured to fuse the first image, the first denoising result, and the second denoising result based on the edge information to obtain a target image.

[0113] As can be seen from the above embodiment, in the embodiment, a flat region and a non-flat region in a first image are determined according to pixel values of at least one color channel of each pixel point in the first image to be processed, color noise reduction processing is performed on the flat region according to a first denoising intensity to obtain a first denoising result, and color noise reduction processing is performed on the non-flat region according to a second denoising intensity to obtain a second denoising result, wherein the first denoising intensity is greater than the second denoising intensity, edge information of the first image is obtained, and the first image, the first denoising result, and the second denoising result are fused based on the edge information to obtain a target image.

[0114] As can be seen, in the embodiment of the present application, since pixel changes in the flat region of the image are mainly caused by noise, the denoising intensity can be increased when the flat region is denoised, so that color noise in the flat region can be removed more cleanly. Since pixel changes in the non-flat region of the image are mainly caused by texture or edges, and noise only accounts for a small part, the denoising intensity can be reduced when the non-flat region is denoised, so that color noise in the non-flat region can be removed more cleanly while the texture and edge features of the image are maintained, avoiding disappearance or distortion of the texture and edges of the image. Since color overflow usually occurs at the junction position of regions with the same brightness and different colors, and the edge information of the image can reflect the aforementioned junction position, fusing the image and its denoising result based on the edge information of the image can avoid the problem of obvious color overflow while removing color noise in the image more cleanly, improving the image denoising effect.

[0115] Optionally, as one embodiment, the determining module 601 can include a first determining module 6011 and a second determining module 6012.

[0116] a first calculating sub-module, configured to calculate, for each pixel point in the first image to be processed, a variance value of a first neighborhood centered at the pixel point in each color channel, wherein a radius of the first neighborhood is greater than 1;

[0117] a first determining sub-module, configured to select a maximum variance value from the variance values of the color channels, and determine that a region where the pixel point is located is a non-flat region if the maximum variance value is greater than a first threshold value;

[0118] a second determining sub-module, configured to determine that the region where the pixel point is located is a flat region if the maximum variance value is less than or equal to the first threshold value.

[0119] Optionally, as an embodiment, the noise reduction module 602 can include:

[0120] a first converting sub-module, configured to convert the first image to a YUV color space to obtain original U channel values and original V channel values corresponding to each pixel point of the first image;

[0121] a second calculating sub-module, configured to calculate, for each pixel point of the flat region, a noise reduction U channel value corresponding to the pixel point according to original U channel values of a second neighborhood centered at the pixel point, and calculate a noise reduction V channel value corresponding to the pixel point according to original V channel values of the second neighborhood centered at the pixel point;

[0122] a third calculating sub-module, configured to calculate, for each pixel point of the non-flat region, a noise reduction U channel value corresponding to the pixel point according to original U channel values of a third neighborhood centered at the pixel point, and calculate a noise reduction V channel value corresponding to the pixel point according to original V channel values of the third neighborhood centered at the pixel point;

[0123] wherein the radius of the second neighborhood and the radius of the third neighborhood are both greater than 1, and the radius of the second neighborhood is greater than the radius of the third neighborhood.

[0124] Optionally, as an embodiment, the second calculating sub-module can include:

[0125] a first calculating unit, configured to calculate a median value or a mean value of the original U channel values of the second neighborhood centered at the pixel point for each pixel point of the flat region, and determine the calculation result as the noise reduction U channel value corresponding to the pixel point;

[0126] a second calculating unit, configured to calculate a median value or a mean value of the original V channel values of the second neighborhood centered at the pixel point, and determine the calculation result as the noise reduction V channel value corresponding to the pixel point.

[0127] Optionally, as an embodiment, the third calculation sub-module can comprise:

[0128] a third calculation unit, configured to calculate, for each pixel point of the non-flat region, a first difference value between the original U channel value corresponding to the pixel point and the original U channel value corresponding to another pixel point in the third neighborhood except the pixel point, and a second difference value between the original V channel value corresponding to the pixel point and the original V channel value corresponding to the another pixel point;

[0129] a fourth calculation unit, configured to calculate the median value of the original U channel value corresponding to the pixel point and the original U channel value of a reference pixel point, and determine the calculation result as the denoised U channel value corresponding to the pixel point, and calculate the median value of the original V channel value corresponding to the pixel point and the original V channel value of the reference pixel point, and determine the calculation result as the denoised V channel value corresponding to the pixel point;

[0130] wherein the reference pixel point is the another pixel point in the third neighborhood with both the first difference value and the second difference value less than the second threshold value.

[0131] Optionally, as an embodiment, the radius of the third neighborhood is determined by the following way:

[0132] for each pixel point of the non-flat region, determining a target color channel with the maximum variance value of a second neighborhood centered at the pixel point;

[0133] calculating the target variance value of a candidate neighborhood centered at the pixel point with a radius of Rs in the target color channel, wherein Rs = R - 1, and R is the radius of the second neighborhood;

[0134] if the target variance value is less than or equal to a third threshold value, then the Rs is determined as the radius of the third neighborhood; if the target variance value is greater than the third threshold value, then the radius of the candidate neighborhood is continuously reduced in the same step-down manner until the target variance value of the candidate neighborhood in the target color channel is less than the third threshold value, or the radius of the candidate neighborhood is 1, and the final reduced radius is determined as the radius of the third neighborhood.

[0135] Optionally, as an embodiment, the fusion module 604 can comprise:

[0136] an operation sub-module, configured to perform weighted operation on the original U channel value and the denoised U channel value corresponding to each pixel point of the first image based on the edge information, to obtain the target U channel value corresponding to each pixel point of the first image; and perform weighted operation on the original V channel value and the denoised V channel value corresponding to each pixel point of the first image, to obtain the target V channel value corresponding to each pixel point of the first image.

[0137] filtering sub-module, configured to perform directional filtering on the target U channel value and the target V channel value corresponding to each pixel point of the first image based on the original U channel value and the original V channel value corresponding to each pixel point of the first image, to obtain a filtering result;

[0138] a second conversion sub-module, configured to perform color space conversion on the filtering result to obtain a target image in the same color space as the first image.

[0139] Any one of the steps in the embodiments of the image denoising method and the specific operations in any one of the steps can be completed by the corresponding modules in the image denoising device. The processes of the corresponding operations completed by the modules in the image denoising device are described with reference to the processes of the corresponding operations in the embodiments of the image denoising method.

[0140] For the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the related parts are described with reference to the parts of the method embodiments.

[0141] Figure 7 is a structural block diagram of an electronic device provided by the embodiments of the present application. The electronic device includes a processing component 722, which further includes one or more processors, and a memory resource represented by a memory 732, for storing instructions, such as an application program, executable by the processing component 722. The application program stored in the memory 732 can include one or more than one module each corresponding to a set of instructions. In addition, the processing component 722 is configured to execute the instructions to perform the method described above.

[0142] The electronic device can also include a power supply component 726 configured to perform power management of the electronic device, a wired or wireless network interface 750 configured to connect the electronic device to a network, and an input / output (I / O) interface 758. The electronic device can operate based on an operating system stored in the memory 732, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM or the like.

[0143] According to still another embodiment of the present application, the present application also provides a computer readable storage medium having stored thereon computer programs / instructions, which, when executed by a processor, implement the steps of the image denoising method according to any one of the above embodiments.

[0144] According to still another embodiment of the present application, the present application also provides a computer program product comprising computer programs / instructions which, when executed by a processor, implement the steps of the image denoising method according to any one of the above embodiments.

[0145] The various embodiments in the present specification are described in progressive manner, and each embodiment focuses on the difference from other embodiments, and the same or similar parts among the various embodiments can be mutually referred to.

[0146] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, device, or computer program product. Therefore, the embodiments of the present application can adopt a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present application can adopt a computer program product in the form of being implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0147] The embodiments of the present application are described with reference to flowcharts and / or block diagrams according to the method, terminal device (system), and computer program product of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the computer or other programmable data processing terminal device produce the functions described in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The apparatus that performs the functions specified in one or more flows and / or blocks.

[0148] These computer program instructions can also be stored in a computer readable memory capable of directing the computer or other programmable data processing terminal device to work in a specific manner, so that the instructions stored in the computer readable memory produce a product comprising instruction apparatus, which implements the functions described in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks Figure 1 The apparatus that performs the functions specified in one or more flows and / or blocks.

[0149] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to the embodiments once they know the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.

[0150] Finally, it is to be understood that the phraseology or terminology such as "first" and "second" etc. used herein is merely intended to differentiate one entity or operation from another entity or operation, without necessarily requiring or implying any actual such relationship or order between such entities or operations. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0151] The above provides a kind of image noise reduction method, electronic equipment and storage medium provided by the present application, have carried out detailed introduction, the principle and implementation mode of the present application are described in this paper by specific example, the above example is only for helping to understand the method of the present application and its core idea;For the person skilled in the art, according to the idea of the present application, there will be changes in specific implementation mode and application range, as described above, the content of the specification should not be understood as the limitation of the present application.

Claims

1. An image denoising method, characterized in that, The method comprises: determining flat regions and non-flat regions in a first image to be processed according to pixel values of at least one color channel of each pixel point in the first image; performing color noise reduction on the flat regions to obtain a first noise reduction result according to a first noise reduction intensity, and performing color noise reduction on the non-flat regions to obtain a second noise reduction result according to a second noise reduction intensity, wherein the first noise reduction intensity is greater than the second noise reduction intensity; obtaining edge information of the first image; fusing the first image, the first noise reduction result and the second noise reduction result based on the edge information to obtain a target image; the performing color noise reduction on the non-flat regions to obtain a second noise reduction result according to a second noise reduction intensity comprises: converting the first image to a YUV color space to obtain original U channel values and original V channel values corresponding to each pixel point of the first image; for each pixel point of the non-flat regions, calculating a noise reduction U channel value corresponding to the pixel point according to original U channel values of a third neighborhood centered on the pixel point, and calculating a noise reduction V channel value corresponding to the pixel point according to original V channel values of the third neighborhood centered on the pixel point, wherein a radius of the third neighborhood is greater than 1; the radius of the third neighborhood is determined by the following way: for each pixel point of the non-flat regions, determining a target color channel with a maximum variance value of a second neighborhood centered on the pixel point; calculating a target variance value of a candidate neighborhood with a radius of Rs in the target color channel, wherein Rs = R-1, R is a radius of the second neighborhood; the second neighborhood is a neighborhood used when performing noise reduction on the flat regions; if the target variance value is less than or equal to a third threshold value, the Rs is determined as a radius of a third neighborhood; if the target variance value is greater than the third threshold value, the radius of the candidate neighborhood is continuously reduced in the same step-down manner until the target variance value of the candidate neighborhood in the target color channel is less than the third threshold value or the radius of the candidate neighborhood is 1, and a final reduced radius is determined as the radius of the third neighborhood.

2. The method of claim 1, wherein, the determining flat regions and non-flat regions in a first image to be processed according to pixel values of at least one color channel of each pixel point in the first image comprises: for each pixel point in the first image to be processed, calculating variance values of each color channel of a first neighborhood centered on the pixel point, wherein a radius of the first neighborhood is greater than 1; selecting a maximum variance value from the variance values of the color channels, if the maximum variance value is greater than a first threshold value, determining that a region where the pixel point is located is a non-flat region; if the maximum variance value is less than or equal to the first threshold value, determining that the region where the pixel point is located is a flat region.

3. The method of claim 1, wherein, the performing color noise reduction on the flat regions to obtain a first noise reduction result according to a first noise reduction intensity comprises: For each pixel point of the flat region, a denoised U channel value corresponding to the pixel point is calculated according to original U channel values of a second neighborhood centered at the pixel point; and a denoised V channel value corresponding to the pixel point is calculated according to original V channel values of the second neighborhood centered at the pixel point. The radius of the second neighborhood is greater than 1, and the radius of the second neighborhood is greater than the radius of the third neighborhood.

4. The method of claim 3, wherein, The calculation of the denoised U channel value and the denoised V channel value for each pixel point of the flat region according to the original U channel values and the original V channel values of the second neighborhood centered at the pixel point includes: For each pixel point of the flat region, a median value or a mean value of the original U channel values of the second neighborhood centered at the pixel point is calculated, and the calculation result is determined as the denoised U channel value corresponding to the pixel point. A median value or a mean value of the original V channel values of the second neighborhood centered at the pixel point is calculated, and the calculation result is determined as the denoised V channel value corresponding to the pixel point.

5. The method of claim 1, wherein, The calculation of the denoised U channel value and the denoised V channel value for each pixel point of the non-flat region according to the original U channel values and the original V channel values of the third neighborhood centered at the pixel point includes: For each pixel point of the non-flat region, a first difference value between the original U channel value corresponding to the pixel point and the original U channel values corresponding to other pixel points in the third neighborhood except the pixel point is calculated, and a second difference value between the original V channel value corresponding to the pixel point and the original V channel values corresponding to the other pixel points is calculated; A median value of the original U channel value corresponding to the pixel point and the original U channel value of a reference pixel point is calculated, and the calculation result is determined as the denoised U channel value corresponding to the pixel point; and a median value of the original V channel value corresponding to the pixel point and the original V channel value of the reference pixel point is calculated, and the calculation result is determined as the denoised V channel value corresponding to the pixel point. The reference pixel point is the other pixel point in the third neighborhood whose first difference value and second difference value are less than a second threshold.

6. The method according to any one of claims 1, 3-5, characterized in that, The fusion of the first image, the first denoised result and the second denoised result based on the edge information to obtain a target image includes: Based on the edge information, a weighted operation is performed on the original U channel value and the denoised U channel value corresponding to each pixel point of the first image to obtain a target U channel value corresponding to each pixel point of the first image; and a weighted operation is performed on the original V channel value and the denoised V channel value corresponding to each pixel point of the first image to obtain a target V channel value corresponding to each pixel point of the first image; Based on the original U channel value and the original V channel value corresponding to each pixel point of the first image, a guided filter processing is performed on the target U channel value and the target V channel value corresponding to each pixel point of the first image to obtain a filter result; and performing color space conversion on the filtering result to obtain a target image in the same color space as the first image.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program comprises instructions that, when executed by the processor, cause the electronic device to perform the method of any one of claims 1-6. The processor executes the computer program to implement the method of any one of claims 1-6.

8. A computer readable storage medium having stored thereon computer programs / instructions, characterized in that, The computer program / instruction is executed by the processor to implement the method of any one of claims 1-6.

9. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instruction is executed by the processor to implement the method of any one of claims 1-6. The computer program / instruction is executed by the processor to implement the method of any one of claims 1-6.

Images