Image noise reduction methods, devices and electronic equipment

Abstract

This application provides an image noise reduction method, apparatus, and electronic device. The method includes: acquiring full-resolution first image data; determining low-resolution first color difference data corresponding to the first image data, wherein the first color difference data is used to characterize the color difference between brightness and color; performing noise reduction processing on the first color difference data to obtain second color difference data; and performing resolution restoration and color difference restoration on the second color difference data to obtain full-resolution second image data, thereby improving the color noise reduction effect.

Description

Technical Field

[0001] This application relates to image processing technology, and more particularly to an image noise reduction method, apparatus, and electronic device. Background Technology

[0002] Most color filters used in imaging devices are arrays that can pass through specific wavelengths such as red, green, blue, yellow, and white. Common examples include RGGB, RYYB, and HexW arrays. After color interpolation (or demosaicing) is performed on the array data, full-resolution RGB, RYB, and RGBW image data is obtained.

[0003] Because the original array data is highly noisy, the noise is still retained after color interpolation, especially the color noise in the R, G, and B channels at full resolution. Related techniques that directly remove color noise from the R, G, and B channels have very limited effect on reducing color noise. Summary of the Invention

[0004] This application provides an image noise reduction method, apparatus, and electronic device, which improves the color noise reduction effect.

[0005] In a first aspect, embodiments of this application provide an image noise reduction method, including:

[0006] Acquire first image data at full resolution;

[0007] Determine the low-resolution first color difference data corresponding to the first image data, wherein the first color difference data is used to characterize the color difference between brightness and color;

[0008] The first color difference data is subjected to noise reduction processing to obtain the second color difference data;

[0009] Resolution restoration and color difference restoration are performed on the second color difference data to obtain full-resolution second image data.

[0010] In one embodiment, the noise reduction processing of the first color difference data includes:

[0011] Each pixel in the first color difference data is filtered according to the noise reduction coefficient corresponding to each pixel in the first color difference data.

[0012] In one implementation, it further includes:

[0013] The noise reduction coefficient corresponding to each pixel is determined based on the low-resolution third image data corresponding to the first image data, wherein the resolution of the third image data is the same as that of the first color difference data.

[0014] In one implementation, determining the noise reduction coefficient corresponding to each pixel based on the third image data corresponding to the first image data includes:

[0015] Based on the brightness data in the third image data, determine the overexposed areas in the third image data;

[0016] The noise reduction coefficient corresponding to each pixel is determined based on the overexposed area.

[0017] In one implementation, determining the noise reduction coefficient corresponding to each pixel based on the third image data corresponding to the first image data includes:

[0018] Determine the gradient of each pixel in the third image data;

[0019] The noise reduction coefficient corresponding to each pixel is determined based on the gradient of each pixel.

[0020] In one implementation, determining the gradient of each pixel in the third image data corresponding to the first image data includes:

[0021] The brightness gradient is determined based on the color data in the window corresponding to each pixel in the third image data, and the chromaticity gradient is determined based on the color difference in the window.

[0022] The gradient of each pixel is determined based on the brightness gradient and the chromaticity gradient.

[0023] In one implementation, determining the noise reduction coefficient corresponding to each pixel based on the third image data corresponding to the first image data includes:

[0024] Determine the local continuous gradient sum of each pixel in the third image data, and the color difference in the window corresponding to each pixel. The local continuous gradient sum includes the horizontal gradient sum and / or the vertical gradient sum.

[0025] Based on the local continuous gradient and the color difference, the high-frequency texture gradient map of the third image data is determined;

[0026] The noise reduction coefficient corresponding to each pixel is determined based on the high-frequency texture gradient map.

[0027] In one implementation, determining the high-frequency texture gradient map of the third image data based on the local continuous gradient and the color difference includes:

[0028] If the difference between the maximum and minimum color difference values ​​in the window corresponding to the pixel is less than a preset threshold, then the value of the local continuous gradient sum of the pixel is set to zero, and the updated local continuous gradient sum of each pixel is determined as the high-frequency texture gradient map of the third image data.

[0029] In one implementation, determining the low-resolution first color difference data corresponding to the first image data includes:

[0030] Based on the differences between the brightness and color data of each pixel in the first image data, the third color difference data corresponding to the full resolution of the first image data is determined;

[0031] The third color difference data is downsampled to obtain the first color difference data.

[0032] In one embodiment, the step of performing resolution restoration and color difference restoration on the second color difference data to obtain full-resolution second image data includes:

[0033] The second color difference data is upsampled to obtain the fourth color difference data at full resolution;

[0034] Based on the brightness data in the first image data, the color data corresponding to each pixel in the fourth color difference data is determined to obtain the second image data corresponding to the fourth color difference data.

[0035] In one implementation, determining the low-resolution first color difference data corresponding to the first image data includes:

[0036] The first image data is downsampled to obtain a low-resolution third image data;

[0037] The first color difference data is determined based on the difference between the brightness data and color data in the third image data.

[0038] In one embodiment, the step of performing resolution restoration and color difference restoration on the second color difference data to obtain full-resolution second image data includes:

[0039] Based on the brightness data in the third image data, the color data corresponding to each pixel in the second color difference data is determined to obtain the low-resolution fourth image data corresponding to the second color difference data.

[0040] The fourth image data is upsampled to obtain the second image data.

[0041] In one embodiment, before determining the low-resolution first color difference data corresponding to the first image data, the method further includes:

[0042] The first image data is subjected to noise normalization processing;

[0043] Also includes:

[0044] The second image data is subjected to inverse noise normalization.

[0045] Secondly, this application provides an image noise reduction apparatus, comprising:

[0046] The acquisition module is used to acquire the first image data at full resolution;

[0047] The first determining module is used to determine the low-resolution first color difference data corresponding to the first image data, wherein the first color difference data is used to characterize the color difference between brightness and color.

[0048] The noise reduction module is used to perform noise reduction processing on the first color difference data to obtain the second color difference data;

[0049] The recovery module is used to perform resolution recovery and color difference recovery on the second color difference data to obtain full-resolution second image data.

[0050] In one implementation, the noise reduction module is used for:

[0051] Each pixel in the first color difference data is filtered according to the noise reduction coefficient corresponding to each pixel in the first color difference data.

[0052] In one implementation, it further includes:

[0053] The second determining module is used to determine the noise reduction coefficient corresponding to each pixel based on the low-resolution third image data corresponding to the first image data, wherein the resolution of the third image data is the same as that of the first color difference data.

[0054] In one implementation, the second determining module is used to:

[0055] Based on the brightness data in the third image data, determine the overexposed areas in the third image data;

[0056] The noise reduction coefficient corresponding to each pixel is determined based on the overexposed area.

[0057] In one implementation, the second determining module is used to:

[0058] Determine the gradient of each pixel in the third image data;

[0059] The noise reduction coefficient corresponding to each pixel is determined based on the gradient of each pixel.

[0060] In one implementation, the second determining module is used to:

[0061] The brightness gradient is determined based on the color data in the window corresponding to each pixel in the third image data, and the chromaticity gradient is determined based on the color difference in the window.

[0062] The gradient of each pixel is determined based on the brightness gradient and the chromaticity gradient.

[0063] In one implementation, the second determining module is used to:

[0064] Determine the local continuous gradient sum of each pixel in the third image data, and the color difference in the window corresponding to each pixel. The local continuous gradient sum includes the horizontal gradient sum and / or the vertical gradient sum.

[0065] Based on the local continuous gradient and the color difference, the high-frequency texture gradient map of the third image data is determined;

[0066] The noise reduction coefficient corresponding to each pixel is determined based on the high-frequency texture gradient map.

[0067] In one implementation, the second determining module is used to:

[0068] If the difference between the maximum and minimum color difference values ​​in the window corresponding to the pixel is less than a preset threshold, then the value of the local continuous gradient sum of the pixel is set to zero, and the updated local continuous gradient sum of each pixel is determined as the high-frequency texture gradient map of the third image data.

[0069] In one implementation, the first determining module is configured to:

[0070] Based on the differences between the brightness and color data of each pixel in the first image data, the third color difference data corresponding to the full resolution of the first image data is determined;

[0071] The third color difference data is downsampled to obtain the first color difference data.

[0072] In one implementation, the recovery module is used for:

[0073] The second color difference data is upsampled to obtain the fourth color difference data at full resolution;

[0074] Based on the brightness data in the first image data, the color data corresponding to each pixel in the fourth color difference data is determined to obtain the second image data corresponding to the fourth color difference data.

[0075] In one implementation, the first determining module is configured to:

[0076] The first image data is downsampled to obtain a low-resolution third image data;

[0077] The first color difference data is determined based on the difference between the brightness data and color data in the third image data.

[0078] In one implementation, the recovery module is used for:

[0079] Based on the brightness data in the third image data, the color data corresponding to each pixel in the second color difference data is determined to obtain the low-resolution fourth image data corresponding to the second color difference data.

[0080] The fourth image data is upsampled to obtain the second image data.

[0081] In one implementation, it further includes:

[0082] The normalization module is used to perform noise normalization processing on the first image data;

[0083] Also includes:

[0084] The inverse normalization module is used to perform noise inverse normalization processing on the second image data.

[0085] Thirdly, this application provides an electronic device, including a memory and a processor, wherein the memory and the processor are connected;

[0086] The memory is used to store computer programs;

[0087] The processor is configured to implement the method as described in the first aspect when the computer program is executed.

[0088] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0089] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0090] This application provides an image noise reduction method, apparatus, and electronic device that performs noise reduction on low-resolution color difference data. By using brightness information to remove color noise from the color data, brightness details can be effectively preserved, and brightness information can be avoided from being over-smoothed. Low-resolution data can effectively remove low-frequency color noise, while reducing subsequent computation costs and computation time, thus improving the color noise reduction effect. Attached Figure Description

[0091] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0092] Figure 1 A schematic flowchart of an image noise reduction method provided in an embodiment of this application;

[0093] Figure 2 A schematic diagram of the variance before and after noise normalization is provided for an embodiment of this application;

[0094] Figure 3 A horizontal gradient and schematic diagram provided for embodiments of this application;

[0095] Figure 4 A vertical gradient and schematic diagram provided for embodiments of this application;

[0096] Figure 5 A schematic diagram of the filtering direction provided in an embodiment of this application;

[0097] Figure 6 This is a schematic diagram of the structure of an image noise reduction device provided in an embodiment of this application;

[0098] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0099] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0100] Considering that color noise in an image tends to be low-frequency after color interpolation, and that directly removing color noise from the R, G, and B channels has very limited effect, this application proposes to obtain corresponding low-resolution color difference information based on full-resolution image data, perform noise reduction on the low-resolution color difference information, and use brightness information to remove color noise from the color data. This can effectively preserve brightness details and effectively remove low-frequency color noise, thereby improving the color noise reduction effect.

[0101] The image denoising method provided in this application will now be described in detail through specific embodiments. It is understood that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0102] Figure 1 This is a schematic flowchart illustrating an image noise reduction method provided in an embodiment of this application. Figure 1 As shown, the method includes:

[0103] S101, Obtain the first image data at full resolution.

[0104] The first image data is full-resolution image data obtained by color interpolation of the array data of the imaging device. In this embodiment, the array mode of the imaging device is not limited; for example, RGGB array, RYYB array, RGBW array, Hex-W array, etc., are all acceptable. The first image data can be full-resolution RGB data, RYB data, RGBW data, etc., and may also include data obtained by format conversion of the aforementioned full-resolution image data. For example, the first image data may include full-resolution RGB data and YUV data obtained by format conversion of the RGB data.

[0105] S102. Determine the low-resolution first color difference data corresponding to the first image data. The first color difference data is used to characterize the color difference between brightness and color.

[0106] The first image data includes the luminance and color data of each pixel. For example, the first image data is RGBW data, with luminance data as W and color data as R, G, and B. The luminance data can be Y after converting RGB data to YUV data. The first color difference data is the color difference between luminance and color. For example, the first image data is RGBW data, with the first color difference data as D1 = WR, D2 = WG, and D3 = WB.

[0107] The resolution of the first color difference data is lower than that of the first image data. Optionally, based on the differences between the brightness and color data of each pixel in the first image data, the third color difference data corresponding to the full resolution of the first image data is determined; the third color difference data is then downsampled to obtain the first color difference data. That is, the color difference data of each pixel in the first image data is first calculated to obtain the third color difference data at full resolution, and then the third color difference data is downsampled to obtain the first color difference data at low resolution.

[0108] Optionally, the first image data is downsampled to obtain low-resolution third image data; the first color difference data is determined based on the difference between the brightness and color data in the third image data. Taking the first image data as full-resolution RGBW data as an example, the third image data is low-resolution RGBW data obtained by downsampling the full-resolution RGBW data. That is, the first image data is first downsampled to obtain low-resolution third image data, and then the color difference data of each pixel in the third image data is calculated to obtain low-resolution first color difference data. When downsampling the first image data or the third color difference data, one layer of downsampling or multiple layers of downsampling can be performed to obtain multi-resolution information. The downsampling method can be Laplacian pyramid, downsampling decomposition, etc., which are not limited in this embodiment.

[0109] S103. Perform noise reduction processing on the first color difference data to obtain the second color difference data.

[0110] In this step, color noise reduction is performed based on the first color difference data, that is, the color difference information between the brightness and color of each pixel. One or more filtering methods can be used when performing noise reduction. This application embodiment does not limit the filtering method.

[0111] Optionally, each pixel in the first color difference data is filtered according to the noise reduction coefficient corresponding to each pixel. The noise reduction coefficient corresponding to each pixel can be different, thereby achieving different degrees of noise reduction for different areas, preserving image details while ensuring noise reduction effect.

[0112] S104. Perform resolution restoration and color difference restoration on the second color difference data to obtain full-resolution second image data.

[0113] After denoising the first color difference data, resolution restoration and color difference restoration need to be performed on the obtained second color difference data to restore it to full-resolution second image data. The second image data is the image data obtained after denoising the first image data.

[0114] Optionally, if in step S102 the third color difference data at full resolution is calculated first, and then the third color difference data is downsampled to obtain the first color difference data, then in this step, the second color difference data is first upsampled to obtain the fourth color difference data at full resolution; then, based on the brightness data in the first image data, the color data corresponding to each pixel in the fourth color difference data is determined to obtain the second image data corresponding to the fourth color difference data. That is, the second color difference data is first upsampled to obtain the fourth color difference data at full resolution, and then the brightness data in the first image data is used to perform color difference recovery on the fourth color difference data to obtain the second image data.

[0115] Optionally, if in step S102 the first image data is downsampled to obtain low-resolution third image data, and then the first color difference data is calculated from the third image data, then in this step, the color data corresponding to each pixel in the second color difference data is determined based on the brightness data in the third image data to obtain low-resolution fourth image data corresponding to the second color difference data; the fourth image data is then upsampled to obtain the second image data. That is, color restoration is first performed on the second color difference data based on the brightness data in the third image data to obtain the fourth image data, and then resolution restoration is performed on the fourth image data to obtain the second image data.

[0116] The upsampling method in this step corresponds to the downsampling method in step S102. For example, if wavelet decomposition is used for downsampling in step S102, then wavelet reconstruction is used for upsampling in this step. The color difference recovery method also corresponds to the color difference calculation method in step S102. For example, if the first color difference data is obtained by subtracting the color data from the luminance data, then the luminance data needs to be subtracted from the noise-reduced color difference data to obtain the recovered color data. The luminance data and the recovered color data constitute the second image data after removing color noise.

[0117] In this embodiment, based on full-resolution image data, the color difference data between corresponding low-resolution brightness and color is determined. Noise reduction is performed on the low-resolution color difference data. By using brightness information to remove color noise in the color data, brightness details can be effectively preserved, avoiding smoothing of brightness information. Low-resolution data can effectively remove low-frequency color noise, while reducing subsequent computational costs and time, and improving the color noise reduction effect.

[0118] Based on the above embodiments, considering that the color noise in the first image data may be uneven, which can lead to poor noise reduction results if the noise level is not uniform, for example, when the noise level is inconsistent, removing noise from areas with high noise levels will inevitably cause the details in other areas with low noise levels to be smoothed out, and vice versa. Therefore, in this embodiment, optionally, after obtaining the first image data in S101, noise normalization processing is performed on the first image data. That is, steps S102 and subsequent steps in the above embodiments are all performed on the first image data after noise normalization processing.

[0119] The method of noise normalization in this application embodiment is not limited. For example, noise normalization can be performed by variance-stabilizing transformation (VST), or it can be achieved by dividing each pixel by the noise standard deviation.

[0120] Taking Poisson-Gaussian noise as an example, noise normalization can be performed using VST. When the noise follows a Poisson-Gaussian distribution, it can be expressed by the following formula:

[0121]

[0122] Where x represents noise data, which in this embodiment is the first image data. * For noise-free data, It follows a Poisson distribution. denoted by Gaussian distribution, and a and b are the correlation coefficients of the noise distribution, respectively.

[0123] Through the VST transformation, the non-uniform noise distribution is normalized to a uniform Gaussian distribution with a uniform variance. The VST transformation formula is:

[0124]

[0125] like Figure 2 This is an example of a variance chart before and after noise normalization. Curve 1 represents the noise variance statistics for different regions before noise normalization, and curve 2 represents the noise variance statistics for different regions after noise normalization. The vertical axis represents the magnitude of the noise variance, and the horizontal axis represents different regions in the data. Figure 2 As can be seen, after noise normalization, the noise distribution changes from non-uniform to uniform. Noise normalization can effectively reduce the difficulty and complexity of noise reduction for the whole, while ensuring that noise removal and detail preservation are taken into account in different areas.

[0126] After noise normalization of the first image data, step S102 can be executed to determine the first color difference data corresponding to the first image data.

[0127] Before denoising the first color difference data in S103, optionally, the denoising coefficient corresponding to each pixel can be determined based on the low-resolution third image data corresponding to the first image data, wherein the resolution of the third image data is the same as that of the first color difference data.

[0128] The third image data is obtained by downsampling the first image data. If the third color difference data is determined first based on the first image data in step S102, then both the first image data and the third color difference data are downsampled to obtain the corresponding third image data and first color difference data respectively. Before denoising the first color difference data, the denoising coefficient of each pixel is determined, and the filtering coefficient is determined by the denoising coefficient, thereby determining the denoising intensity.

[0129] Optionally, different methods can be used to specifically determine the noise reduction coefficients for different regions. For example, setting separate noise reduction coefficients for overexposed areas, edge areas, and high-frequency texture areas can effectively improve the noise reduction effect in these areas. These will be explained in detail below.

[0130] Optionally, the processing method for overexposed areas is as follows: determine the overexposed areas in the third image data based on the brightness data in the third image data; determine the noise reduction coefficient corresponding to each pixel based on the overexposed areas.

[0131] Optionally, iterate through each pixel of the third image data. Using the brightness data of each pixel, such as W, determine whether the W pixel value of the current pixel is greater than a threshold. For example, the pixel value of the third image data is between 0 and 1, and the threshold is 0.98. If the W pixel value of a pixel is greater than the threshold, the pixel is considered to be overexposed. Record a flag bit of 1 for overexposed points and a flag bit of 0 for non-overexposed points to generate an overexposed area mask. Set the corresponding noise reduction coefficient for the pixels marked as 1 in the overexposed area mask. For example, set the noise reduction coefficient to less than 1 to reduce the noise reduction intensity of overexposed areas, and set the noise reduction coefficient of non-overexposed areas to 1 to perform normal noise reduction in non-overexposed areas, thereby avoiding the overexposed areas from being smoothed out and preserving more detail information in the overexposed areas.

[0132] Optionally, the processing method for edge regions is as follows: determine the gradient of each pixel in the third image data; determine the noise reduction coefficient corresponding to each pixel based on the gradient of each pixel.

[0133] Edge regions are determined by analyzing the gradient of each pixel, and then the corresponding noise reduction coefficients are determined. Optionally, the brightness gradient is determined based on the color data in the window corresponding to each pixel in the third image data, and the chromaticity gradient is determined based on the color difference in the window; the gradient of each pixel is then determined based on the brightness gradient and the chromaticity gradient.

[0134] For example, the gradient is calculated using the extreme value method:

[0135] E = max i∈win x i -min j∈win x j (3)

[0136] The above formula indicates that the gradient E of a point is the difference between the maximum and minimum values ​​in the window win. However, in this embodiment, for a pixel in the third image data, the gradient E is calculated based on the R, G, and B color data of each pixel in the corresponding window. R E G E B Then, the maximum value is taken as the brightness gradient E of that pixel. le=max(E R E G E B ).

[0137] Similarly, the gradient E is calculated based on the color differences (RG), (GB), and (BR) of each pixel in the window corresponding to that pixel. R-G E G-B E B-R Then, the maximum value is taken as the chromaticity gradient R of that pixel. ce =max(E R-G E G-B E B-R Finally, the gradient E of that pixel is obtained. final =E le ×E ce .

[0138] Based on the gradient of each pixel, edge regions in the third image data can be determined. Regions with larger gradient values ​​are edge regions, while regions with smaller gradient values ​​are flat regions. Different gradient values ​​correspond to different denoising coefficients (the mapping relationship between gradient values ​​and denoising coefficients can be pre-established). By setting the corresponding denoising coefficient for each pixel based on the gradient value, targeted denoising for different regions can be achieved. For example, for pixels with large gradient values, the denoising intensity can be reduced by setting a denoising coefficient, such as setting it to a value less than 1. This allows for effective color denoising of the edge areas while avoiding edge color overflow.

[0139] Optionally, the processing method for high-frequency texture regions is as follows: determine the local continuous gradient sum of each pixel in the third image data, and the color difference in the window corresponding to each pixel. The local continuous gradient sum includes the horizontal gradient sum and / or the vertical gradient sum; determine the high-frequency texture gradient map of the third image data based on the local continuous gradient sum and the color difference; determine the noise reduction coefficient corresponding to each pixel based on the high-frequency texture gradient map.

[0140] High-frequency texture regions are detected and processed accordingly by calculating the sum of continuous gradients at each pixel. Optionally, the horizontal gradient sum is as follows: Figure 3 As shown, the dashed box represents the horizontal gradient and the corresponding sliding window. The window size can be set as needed. The horizontal gradient within the window is Sh = |W2-W1| + |W3-W2| + |W4-W3|. The window slides through all pixels, and the horizontal gradient Gh of each pixel is the sum of the horizontal gradients and Sh of all windows that have slid through that pixel.

[0141] Optional, vertical gradient and such Figure 4As shown, the dashed box represents the vertical gradient and the corresponding window. The window size can be set as needed. The vertical gradient and Sv within the window are Sv = |W21-W11| + |W22-W12| + |W23-W13| + |W24-W14|. The window slides through all pixels, and the vertical gradient and Gv of each pixel are the sum of the vertical gradients and Sv of all windows that have slid through that pixel.

[0142] The local continuous gradient sum G of each pixel can be the sum of the horizontal gradient sum and the vertical gradient sum, i.e., G = Gh + Gv. Optionally, the local continuous gradient sum of each pixel can be restricted to the range of 0-1.

[0143] For each pixel, calculate the color difference of each pixel in the window corresponding to the pixel. If the difference between the maximum and minimum color differences in the window corresponding to the pixel is less than a preset threshold, set the value of the local continuous gradient sum of the pixel to zero, and determine the updated local continuous gradient sum of each pixel as the high-frequency texture gradient map of the third image data.

[0144] For example, calculating the color difference (RG) and (GB) for each pixel within a window corresponding to each pixel, if the difference between the maximum and minimum values ​​in (RG) is less than a preset threshold, and the difference between the maximum and minimum values ​​in (GB) is also less than a preset threshold, then the window is considered a region of the same color, and the sum of the local continuous gradients of that pixel is set to zero. The final high-frequency texture gradient map is obtained through this method, and then denoising coefficients are derived based on the gradient map, thus achieving targeted denoising for high-frequency texture regions. Specifically, for pixels in the high-frequency texture gradient map where the sum of the local continuous gradients is zero, the denoising strength can be increased accordingly by adjusting the denoising coefficient. For example, the denoising coefficient can be set to a value greater than 1 to remove color noise between high-frequency textures, thus preserving the color information of the texture while denoising.

[0145] In the above method, different methods are used to determine the noise reduction coefficient of each pixel for noise reduction in overexposed areas, edge areas, and high-frequency texture areas. In practical applications, one or more of these methods can be used to determine the noise reduction coefficient of each pixel. The noise reduction coefficients obtained by each method do not affect each other and can be used together to determine the filtering coefficients in the subsequent noise reduction process.

[0146] Furthermore, as can be seen from the above process of determining the noise reduction coefficient, the first color difference data is not used in the calculation. Therefore, the process of determining the noise reduction coefficient based on the third image data after downsampling the first image data to obtain low-resolution third image data, and the process of determining the first color difference data based on the third image data, do not affect each other. The order of their execution in this embodiment is not limited, or they can be executed in parallel. In addition, the above process of determining the noise reduction coefficient can also be performed before downsampling, that is, the noise reduction coefficient of each pixel is determined based on the first image data, and then downsampling is performed on the determined noise reduction coefficient.

[0147] The following continues to explain the noise reduction process for the first color difference data in S103.

[0148] Optionally, the noise reduction process may include two parts: infinite impulse response (IIR) filtering and finite impulse response (FIR) filtering. IIR filtering has the advantages of low computational cost, speed, and effective removal of low-frequency color noise, while FIR filtering can remove noise from complex texture regions.

[0149] The IIR filtering formula is as follows:

[0150] y[n]=x[n]+α*y[n-1] (4)

[0151] Where x[n] is the noise data, i.e. the first color difference data, y[n] is the output noise reduction data, α is the IIR filter parameter, which is related to the aforementioned noise reduction coefficient, n is the pixel position, n is the current pixel, and n-1 represents the previous pixel.

[0152] According to the above formula, for the first color difference data, horizontal IIR filtering is performed row by row from left to right, and then horizontal IIR filtering is performed row by row from right to left. Similarly, IIR filtering is also performed in the vertical direction and along the positive and negative diagonal directions. Thus, as... Figure 5 As shown, IIR filtering is performed in eight directions.

[0153] In addition, after completing IIR filtering in each direction, a normalization operation is required. This can be achieved by performing the same IIR filtering operation on equal-sized sets of all "1"s to obtain the normalization coefficients. The final normalized output is:

[0154]

[0155] Where IIR[x] represents IIR filtering on pixel x[n], and IIR[1] represents the same IIR filtering on “1”.

[0156] After completing IIR filtering in all eight directions, FIR filtering is then performed. Optionally, a simplified version of FIR filtering can be used, such as a simplified version of Non-Local Means (NLM), to reduce computational cost and processing time. The original NLM formula is shown below:

[0157]

[0158] in B(p,r s () represents a center point of p and a radius of r. s The search window. It represents the similarity between two pixels p and q, and h is the filtering parameter, which is related to the aforementioned noise reduction coefficient. The center point is located at p, and the radius is r. k Window N(p,r) k The pixel value vector within ) The center point is located at q, and the radius is r. k Window N(q,r) k The pixel value vector within ).

[0159] In this embodiment of the application, the search window B(p,r) is used. s The data is restricted to a single row and slides with a fixed step size (greater than 1). For a window N(p,r) k ), N(q,r k It is also limited to a single line. In addition, integral graph acceleration is performed for NLM to achieve a simplified acceleration of NLM.

[0160] It should be noted that the above description only uses IIR filtering and FIR filtering as examples, but is not limited to these. The filtering methods in the embodiments of this application are not limited.

[0161] After denoising the first color difference data to obtain the second color difference data, S104 can be executed to restore the resolution and color difference of the second color difference data, thus obtaining the second image data with full resolution.

[0162] Optionally, if noise normalization was performed on the first image data in the aforementioned steps, then after obtaining the second image data, inverse noise normalization is also required to restore the original information distribution. Inverse noise normalization is the inverse operation of the aforementioned noise normalization. For example, if VST was used for noise normalization, then inverse VST is performed here:

[0163]

[0164] Where y is the second image data, and a and b are the correlation coefficients of the aforementioned noise distribution.

[0165] By employing noise normalization and inverse noise normalization processes, the method provided in this application embodiment is applicable not only to uniform noise data but also to non-uniform noise data.

[0166] It should also be noted that, in the embodiments of this application, when downsampling the first image data or the third color difference data, multiple downsampling layers can be performed. Correspondingly, when restoring resolution, multiple upsampling layers are required. In addition, noise reduction processing can be performed on the first color difference data at each resolution layer to improve the noise reduction effect.

[0167] Figure 6 This is a schematic diagram of the structure of an image synthesis device provided in an embodiment of this application, as shown below. Figure 4 As shown, the image noise reduction device 600:

[0168] Acquisition module 601 is used to acquire first image data at full resolution;

[0169] The first determining module 602 is used to determine the low-resolution first color difference data corresponding to the first image data, wherein the first color difference data is used to characterize the color difference between brightness and color.

[0170] The noise reduction module 603 is used to perform noise reduction processing on the first color difference data to obtain the second color difference data;

[0171] The recovery module 604 is used to perform resolution recovery and color difference recovery on the second color difference data to obtain full-resolution second image data.

[0172] In one implementation, the noise reduction module 603 is used for:

[0173] Each pixel in the first color difference data is filtered according to the noise reduction coefficient corresponding to each pixel in the first color difference data.

[0174] In one implementation, it further includes:

[0175] The second determining module is used to determine the noise reduction coefficient corresponding to each pixel based on the low-resolution third image data corresponding to the first image data, wherein the resolution of the third image data is the same as that of the first color difference data.

[0176] In one implementation, the second determining module is used to:

[0177] Based on the brightness data in the third image data, determine the overexposed areas in the third image data;

[0178] The noise reduction coefficient for each pixel is determined based on the overexposed areas.

[0179] In one implementation, the second determining module is used to:

[0180] Determine the gradient of each pixel in the third image data;

[0181] The noise reduction coefficient for each pixel is determined based on the gradient of each pixel.

[0182] In one implementation, the second determining module is used to:

[0183] The brightness gradient is determined based on the color data in the window corresponding to each pixel in the third image data, and the chromaticity gradient is determined based on the color difference in the window.

[0184] The gradient of each pixel is determined based on the luminance gradient and chrominance gradient.

[0185] In one implementation, the second determining module is used to:

[0186] Determine the local continuous gradient sum of each pixel in the third image data, and the color difference in the window corresponding to each pixel. The local continuous gradient sum includes the horizontal gradient sum and / or the vertical gradient sum.

[0187] The high-frequency texture gradient map of the third image data is determined based on the local continuous gradient and color difference.

[0188] The noise reduction coefficient for each pixel is determined based on the high-frequency texture gradient map.

[0189] In one implementation, the second determining module is used to:

[0190] If the difference between the maximum and minimum color difference in the window corresponding to a pixel is less than a preset threshold, the value of the local continuous gradient sum of the pixel is set to zero, and the updated local continuous gradient sum of each pixel is determined as the high-frequency texture gradient map of the third image data.

[0191] In one implementation, the first determining module 602 is used to:

[0192] Based on the differences between the brightness and color data of each pixel in the first image data, the third color difference data corresponding to the full resolution of the first image data is determined;

[0193] The third color difference data is downsampled to obtain the first color difference data.

[0194] In one implementation, the recovery module 604 is used for:

[0195] The second color difference data is upsampled to obtain the fourth color difference data at full resolution;

[0196] Based on the brightness data in the first image data, the color data corresponding to each pixel in the fourth color difference data is determined to obtain the second image data corresponding to the fourth color difference data.

[0197] In one implementation, the first determining module 602 is used to:

[0198] The first image data is downsampled to obtain a low-resolution third image data;

[0199] The first color difference data is determined based on the difference between the brightness data and color data in the third image data.

[0200] In one implementation, the recovery module 604 is used for:

[0201] Based on the brightness data in the third image data, the color data corresponding to each pixel in the second color difference data is determined to obtain the low-resolution fourth image data corresponding to the second color difference data.

[0202] The fourth image data is upsampled to obtain the second image data.

[0203] In one implementation, it further includes:

[0204] The normalization module is used to perform noise normalization processing on the first image data;

[0205] Also includes:

[0206] The inverse normalization module is used to perform noise inverse normalization processing on the second image data.

[0207] The image synthesis apparatus of this application embodiment can be used to implement the image noise reduction method in the above method embodiment. Its implementation principle and calculation effect are similar, and will not be described again here.

[0208] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 7 As shown, the electronic device 700 includes a memory 701 and a processor 702, which can be connected via a bus 703.

[0209] The memory 701 is used to store computer programs.

[0210] The processor 702 is used to implement the image noise reduction method in the above method embodiments when the computer program is executed.

[0211] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the image noise reduction method described in the above method embodiments.

[0212] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the image noise reduction method described in the above method embodiments.

[0213] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0214] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An image denoising method, characterized in that, The method comprises: acquiring first image data of full resolution; determining first color difference data of low resolution corresponding to the first image data, the first color difference data being used to represent color difference of brightness and color; performing noise reduction processing on the first color difference data to obtain second color difference data; performing resolution recovery and color difference recovery on the second color difference data to obtain second image data of full resolution; the determining of the first color difference data of low resolution corresponding to the first image data comprises: determining third color difference data of full resolution corresponding to the first image data according to differences between brightness data and color data of each pixel point in the first image data; performing down-sampling on the third color difference data to obtain the first color difference data; or performing down-sampling on the first image data to obtain third image data of low resolution; determining the first color difference data according to differences between brightness data and color data in the third image data. the noise reduction processing on the first color difference data comprises:

2. The method of claim 1, wherein, performing filtering on each pixel point in the first color difference data according to a noise reduction coefficient corresponding to the pixel point in the first color difference data. The method further comprises:

3. The method of claim 2, wherein, determining the noise reduction coefficient corresponding to each pixel point according to third image data of low resolution corresponding to the first image data, the resolution of the third image data being the same as that of the first color difference data. The determining of the noise reduction coefficient corresponding to each pixel point according to the third image data corresponding to the first image data comprises:

4. The method of claim 3, wherein, determining an overexposure area in the third image data according to brightness data in the third image data; determining the noise reduction coefficient corresponding to each pixel point according to the overexposure area. The determining of the noise reduction coefficient corresponding to each pixel point according to the third image data corresponding to the first image data comprises:

5. The method of claim 3, wherein, determining a gradient of each pixel point in the third image data; determining the noise reduction coefficient corresponding to each pixel point according to the gradient of the pixel point. The determining of the gradient of each pixel point in the third image data corresponding to the first image data comprises:

6. The method of claim 5, wherein, determining a luminance gradient according to color data in a window corresponding to each pixel point in the third image data, and determining a chroma gradient according to color difference in the window; determining the gradient of each pixel point according to the luminance gradient and the chroma gradient. The determining of the noise reduction coefficient corresponding to each pixel point according to the third image data corresponding to the first image data comprises:

7. The method of claim 3, wherein, determining a local continuous gradient sum of each pixel point in the third image data and color difference in a window corresponding to the pixel point, the local continuous gradient sum comprising a horizontal gradient sum and / or a vertical gradient sum; determining a high-frequency texture gradient map of the third image data according to the local continuous gradient sum and the color difference; determining the noise reduction coefficient corresponding to each pixel point according to the high-frequency texture gradient map. The determining of the high-frequency texture gradient map of the third image data according to the local continuous gradient sum and the color difference comprises:

8. The method of claim 7, wherein, ​ If a difference between a maximum value and a minimum value of the color difference in the window corresponding to the pixel point is less than a preset threshold, a value of a local continuous gradient sum of the pixel point is set to zero, and the updated local continuous gradient sum of each pixel point is determined as a high-frequency texture gradient map of the third image data.

9. The method of claim 1, wherein, The resolution recovery and the color difference recovery on the second color difference data to obtain the full-resolution second image data, comprising: upsampling the second color difference data to obtain fourth color difference data of full resolution; determining color data corresponding to each pixel point in the fourth color difference data according to luminance data in the first image data to obtain the second image data corresponding to the fourth color difference data.

10. The method of claim 1, wherein, The resolution recovery and the color difference recovery on the second color difference data to obtain the full-resolution second image data, comprising: determining color data corresponding to each pixel point in the second color difference data according to luminance data in the third image data to obtain fourth image data of low resolution corresponding to the second color difference data; upsampling the fourth image data to obtain the second image data.

11. The method according to any one of claims 1 to 8, characterized in that, Before the determining the low-resolution first color difference data corresponding to the first image data, the method further comprises: performing noise normalization processing on the first image data; The method further comprises: performing noise inverse normalization processing on the second image data.

12. An image denoising apparatus characterized by comprising: Comprising: an acquisition module, configured to acquire full-resolution first image data; a first determination module, configured to determine low-resolution first color difference data corresponding to the first image data, the first color difference data being used to represent color difference of luminance and color; a noise reduction module, configured to perform noise reduction processing on the first color difference data to obtain second color difference data; a recovery module, configured to perform resolution recovery and color difference recovery on the second color difference data to obtain full-resolution second image data; The first determination module is specifically configured to: determine third color difference data of full resolution corresponding to the first image data according to a difference between luminance data and color data of each pixel point in the first image data; downsample the third color difference data to obtain the first color difference data; or downsample the first image data to obtain third image data of low resolution; determine the first color difference data according to a difference between luminance data and color data in the third image data.

13. An electronic device, comprising: comprising a memory and a processor, the memory and the processor being connected; The memory is used to store a computer program; The processor is used to implement the method of any one of claims 1-11 when the computer program is executed.

14. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1-11.

15. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1-11.

Images