Device and method for denoising raw images based on Bayer format

Abstract

The present invention provides a denoising device and method for a raw image based on a Bayer format. The denoising device calculates the similarity of local neighborhoods of different types through a similarity calculation unit, and the weight self-correction of a weight correction unit can better remove noise. At the same time, a local parameter estimation unit adaptively estimates denoising parameters (i.e., local denoising intensity and local noise back-addition ratio) according to local information of the raw image, thereby improving the denoising effect and being conducive to hardware implementation.

Description

Technical Field

[0001] The present invention relates to image signal processing technology, and in particular to a denoising device and method for a raw image based on a Bayer format. Background Art

[0002] There are many types of image noise in the camera system (hereinafter referred to as Camera), and the causes are also relatively complex.

[0003] From the perspective of the camera system, most consumer cameras on the market currently obtain image information through a photosensitive sensor equipped with a single ColorFilter Array (CFA, color filter array). The sensor directly outputs a raw image. Among them, due to factors such as production cost and algorithm implementation, raw images are usually in Bayer format. The Demosaic algorithm intends to supplement the missing color information by interpolation. With the advancement of optoelectronic systems, today's cameras can output high-quality images in scenes with good exposure conditions. However, in low-light scenes, due to the small number of photons entering the imaging system, the signal-to-noise ratio of the raw image obtained by CFA is very low, causing most cameras to output noise. At the same time, in such scenes, the camera system often adapts to low-level signals by increasing its own sensitivity (ISO level), which further amplifies the output noise. This type of noise usually appears in the form of spots (Splotches), which is very abrupt to the human visual system.

[0004] From a statistical point of view, according to the definition in the European camera test standard EMVA1288, camera noise can be generally divided into two categories: the first category is the statistical fluctuation noise caused by the effective signal that conforms to the Poisson distribution. Since the fluctuation of the number of charges converted by photoelectricity conforms to statistical laws, according to the theory of quantum mechanics, the fluctuation conforms to the Poisson distribution. Therefore, the variance of the fluctuation is equal to the average value of the accumulated charge number, and the corresponding calculation formula (1) is as follows:

[0005]

[0006] in, is the variance of charge number fluctuation; μ e is the average value of the number of charges. The first type of noise is also called shot noise, which is determined by basic physical laws and is universal for any type of camera.

[0007] The second type is the noise that is inherent to the camera itself and has nothing to do with the signal. It is the noise caused by the image sensor readout circuit, camera signal processing and amplification circuit, etc. The inherent noise of each camera is different. The corresponding calculation formula (2) is as follows:

[0008]

[0009] in, is the variance of the noise; The noise caused by Dark Signal; is the quantization noise; μ y is the average value of the digital signal; μ y_dark is the digital value of dark current; K is the overall system gain.

[0010] In addition, for digital cameras, quantization noise is generated when performing analog-to-digital conversion on video signals, and the lower the quantization accuracy, the greater the noise.

[0011] After studying the causes of noise, we can better use the mathematical model of image noise for denoising. Denoising can be performed in the RGB domain after the demosaic algorithm, or in the grayscale or chromaticity domain. So why do we need to perform denoising on the raw image? The main reason is that the noise model of the raw image directly output by the sensor can be described by the above mathematical model. With the mathematical description, the corresponding relationship between signal strength and noise intensity can be established. In theory, we can perform denoising on the image supported by the mathematical model of noise, so that the processed image noise is closer to Gaussian distribution. On the contrary, when the image data of the raw image passes through the various levels of ISP modules, each level of processing will have different effects on the image noise distribution, and the nature of the image noise will be more complicated, making it more difficult to process.

[0012] As for the existing denoising methods, as of now, in addition to the emerging machine learning and deep learning methods with high implementation costs, the traditional denoising methods can be roughly divided into the following categories:

[0013] (1) Filtering: Taking the non-local mean filtering algorithm (NL-means, Buades et al., "Anon_local algorithm for image denoising" published in 2005) as an example, its main idea is that the denoising result of the current pixel point V(i) is obtained by weighting all the pixels V(j) within a certain range around it, where the weight of each V(j) is calculated by the similarity between the neighborhood of each V(j) and the neighborhood of V(i).

[0014] (2) Domain transformation type. Take the wavelet threshold algorithm (Wavelet threshold, "De-noising by soft-thresholding" published by Donoho et al. in 1992) as an example. Its main idea is to perform orthogonal wavelet decomposition on the original noisy image at each scale, retain all decomposition values ​​at large scale (i.e. low resolution), and for the decomposition values ​​at small scale (i.e. high resolution), set a threshold and set the wavelet coefficients below the threshold to zero, while the wavelet coefficients above the threshold are retained or modified to a certain extent, and finally the processed wavelet coefficients are reconstructed by inverse wavelet transform.

[0015] (3) Sparse representation class: Taking the nonlocally centralized sparse representation algorithm (NCSR, "Nonlocally Centralized Sparse Respresentation for Image Restoration" published by Dong et al. in 2011) as an example, its main idea is to use the K-means clustering algorithm to divide each pixel block (Patch) in the image into K categories, and learn a PCA sub-dictionary for each category. For each given patch, the distance between it and all clusters is calculated to determine which cluster the patch belongs to, and then the corresponding PCA sub-dictionary encoding is selected to reconstruct the patch.

[0016] Therefore, how to provide an image denoising algorithm that is applied in consumer cameras, smoothes noisy images while retaining high-frequency effective information as much as possible, and is easy to implement in hardware has become an important problem that needs to be solved. Summary of the invention

[0017] The purpose of the present invention is to provide a denoising device and method for a raw image based on a Bayer format, which can better remove noise compared to the prior art, while also improving the denoising effect of previous traditional methods and being conducive to hardware implementation.

[0018] In order to solve the above problems, the present invention provides a denoising device for a raw image based on a Bayer format, comprising:

[0019] An image data input unit, used for obtaining image data of a raw image based on a Bayer format according to an original noisy image;

[0020] A local parameter estimation unit, used for estimating the local denoising intensity and the local noise adding back ratio corresponding to each pixel point in the raw image according to the image data of the raw image;

[0021] A similarity calculation unit, configured to calculate the similarity between different types of neighborhoods in each preset window area of ​​the raw image according to the image data of the raw image;

[0022] A weight calculation unit, used for calculating the weight of each pixel point in the same channel in each preset window area around each pixel point according to the local denoising strength and similarity;

[0023] A texture calculation unit, used to calculate local texture information in the preset window area around each pixel point according to the weight;

[0024] A weight correction unit, used for correcting the weight according to the local texture information to obtain a new weight;

[0025] A weighted averaging unit, configured to perform weighted averaging on the new weight and each pixel point of the same channel in each preset window area to obtain a denoising result of the current point, thereby obtaining a denoised image; and

[0026] The noise adding back unit is used to obtain a final denoising result according to the denoised image and the local noise adding back ratio, so as to obtain a final denoised image.

[0027] Optionally, the preset window area is a 15x15 area.

[0028] Furthermore, the local parameter estimation unit includes:

[0029] A brightness calculation unit, used for filtering the same-channel pixels of the raw image in each of the preset window areas to obtain a brightness value corresponding to each pixel in the preset window area;

[0030] A first piecewise linear mapping unit, configured to perform 5-segment linear mapping and square root according to the brightness value corresponding to each pixel point in the preset window area, so as to obtain a local denoising intensity corresponding to each pixel point in the preset window area; and

[0031] The second piecewise linear mapping unit is used to perform 5-segment linear mapping according to the brightness value corresponding to each pixel point in the preset window area to obtain the local noise return ratio corresponding to each pixel point in the preset window area.

[0032] Furthermore, the brightness value Luma satisfies the following calculation formula:

[0033] Luma=(Input 15×15

[16] +Input 15×15

[18] +Input 15×15

[20] +Input 15×15

[22]

[0034] +Input 15×15

[24] +Input 15×15

[26] +Input 15×15

[28] +Input 15×15

[46]

[0035] +Input 15×15

[48] +Input 15×15

[50] +Input 15×15

[52] +Input 15×15

[54]

[0036] +Input 15×15

[56] +Input 15×15

[58] +input 15×15

[76] +Input 15×15

[78]

[0037] +input 15×15

[80] +Input 15×15

[84] +Input 15×15

[86] +Input 15×15

[88]

[0038] +Input 15×15

[106] +Input 15×15

[108] +Input 15×15

[116] +Input 15×15

[118]

[0039] +Input 15×15

[136] +Input 15×15

[138] +Input 15×15

[140] +Input 15×15

[144]

[0040] +Input 15×15

[146] +Input 15×15

[148] +Input 15×15

[166] +Input 15×15

[168]

[0041] +Input 15×15

[170] +Input 15×15

[172] +Input 15×15

[174] +Input 15×15

[176]

[0042] +Input15×15

[178] +Input 15×15

[196] +Input 15×15

[198] +Input 15×15

[200]

[0043] +Input 15×15

[202] +Input 15×15

[204] +Input 15×15

[206] +Input 15×15

[208]

[0044] +2

[0045] ×(Input 15×15

[82] +Input 15×15

[110] +Input 15×15

[114] +Input 15×15

[142] )

[0046] +4×Input 15×15

[112] ) / 56

[0047] Among them, Input 15×15 The pixel value of the 15x15 area around each pixel.

[0048] Furthermore, the calculation formula of the estimated value Sigma of the local denoising strength is as follows:

[0049]

[0050] Wherein, Luma is the brightness value of each pixel in the preset window area of ​​the raw image; S s0～4 is the slope of each of the 5 linear mappings; X s0～4 Y is the horizontal coordinate value of the starting point of each segment in the 5-segment linear mapping; s0～4 It is the ordinate value of the starting point of each segment in the 5-segment linear mapping.

[0051] Furthermore, the calculation formula of the estimated value AddbackRatio of the local noise add-back ratio is as follows:

[0052]

[0053] Wherein, Luma is the brightness value of each pixel in the preset window area of ​​the raw image; S a0～4 is the slope of each of the 5 linear mappings; X a0～4 Y is the horizontal coordinate value of the starting point of each segment in the 5-segment linear mapping; a0～4 It is the ordinate value of the starting point of each segment in the 5-segment linear mapping.

[0054] Further, the similarity calculation unit includes 8 neighborhood types set in parallel and a similarity calculation subunit, and the similarity calculation unit calculates a similarity matrix through each of the neighborhood types and the similarity calculation subunit, and finally obtains 8 similarity matrices;

[0055] The eight neighborhood types are the neighborhood of each pixel point in the vertical and horizontal directions and the neighborhood in the diagonal direction.

[0056] Furthermore, each similarity matrix Dist i,j,k The calculation formula is as follows:

[0057]

[0058] Among them, Center is the pixel value of each pixel point and its neighborhood in the preset window area of ​​the raw image; Neighbor is the pixel value of other points in the same channel and its neighborhood in the 15×15 area around each pixel point in the preset window area of ​​the raw image; c is the channel, and when c=0, it is the same channel as the channel and the center pixel point, and when c=1, it is a different channel from the center pixel point; Dist is the similarity matrix, and its size is 7×7×8; i and j are the two-dimensional coordinate positions of the Dist matrix respectively; k is the third-dimensional coordinate position of the Dist matrix.

[0059] Furthermore, the weight calculation unit obtains 8 weight matrices according to the 8 similarity matrices, each of the weight matrices Weight1 i,j,k The calculation formula is as follows:

[0060]

[0061] Among them, Dist is the similarity matrix, and its size is 7×7×8; Sigma is the local denoising intensity corresponding to each pixel point of the raw image in each preset window area; Weight1 is the weight matrix of the same channel points in the 15×15 area around each pixel point, and its size is 7×7×8; e represents the exponential operation; i and j are the two-dimensional coordinate positions of the Dist matrix and the Weight1 matrix respectively; k is the third-dimensional coordinate position of the Dist matrix and the Weight1 matrix.

[0062] Furthermore, the texture calculation unit obtains 8 texture degrees according to the 8 weight matrices, each of which has a texture degree Texture k The calculation formula is as follows:

[0063]

[0064] Among them, Weight1 is the weight matrix of the points in the same channel in the 15×15 area around each pixel, and its size is 7×7×8; HighPassKernel is the parameter of the high-pass filter; Texture is the texture degree in the 15×15 area around each pixel, and its size is 1×8; i and j are the two-dimensional coordinate positions of the Weight1 matrix respectively; k is the third-dimensional coordinate position of the Weight1 matrix.

[0065] Furthermore, the weight correction unit includes a bilateral filtering unit, which is used to perform bilateral filtering on 8 weight matrices output by the weight calculation unit using 8 texture degrees output by the texture calculation unit as smoothing parameters to obtain 8 new weight matrices.

[0066] Furthermore, each new weight matrix Weight2 i,j,k The following calculation formula is satisfied:

[0067] D i,j,k 2 =(X i,j,k -X center ) 2 +(Y i,j,k -Y center ) 2

[0068]

[0069] R i,j,k 2 =(Weight1 i,j,k -Weight1 center,k ) 2

[0070]

[0071]

[0072]

[0073] Among them, X i,j is the horizontal coordinate of the point (i, j) in the 7×7 window in each neighborhood type in the Cartesian coordinate system; i,j D is the ordinate of the point (i, j) in the 7×7 window in each neighborhood type in the Cartesian coordinate system; i,jis the Euclidean distance from the point (i, j) in the 7×7 window to the center of the 7×7 window in each neighborhood type; Texture is the texture degree in the 15×15 area around each pixel, which represents the smoothing strength; GainD is the fixed gain 1 of the smoothing strength; GaussianKernel is the value of the Gaussian kernel, whose size is 7×7×8; Weight is the weight matrix of the points in the same channel in the 15×15 area around each pixel, whose size is 7×7×8; R i,j is the residual of the center point of the point (i, j) in the 7×7 window in each neighborhood type; GainR is the fixed gain of smoothing intensity 2; ResidualKernel is the value of the residual kernel, and its size is 7×7×8; BilateralKernel is the value of the bilateral filter kernel, and its size is 7×7×8; i and j are the two-dimensional coordinate positions of the Weight1 matrix and the Weight2 matrix respectively; k is the third-dimensional coordinate position of the Weight1 matrix and the Weight2 matrix.

[0074] Furthermore, the denoising result NrOut obtained by the weighted average unit satisfies the formula:

[0075]

[0076] Among them, Input 15×15当前点 are the pixels of the same channel in the 15×15 area around each pixel in the raw image (7×7); Weight2 is the weight matrix of the points of the same channel in the 15×15 area around each pixel, and its size is 7×7×8; the size of NrOut is 1×1; i and j are the two-dimensional coordinate positions of the Dist matrix and the Weight2 matrix respectively; k is the third-dimensional coordinate position of the Dist matrix and the Weight2 matrix.

[0077] Furthermore, the final denoising result Output obtained by the noise back-addition unit satisfies the following formula:

[0078] W a =clip(AddbackRatio+Offset,0,1)

[0079] Output=(1-W a )×NrOut+W a ×Input

[0080] Wherein, AddbackRatio is the local noise adding ratio corresponding to each pixel point in each preset area of ​​the raw image; aThe weight for adding back the noise of the raw image; Offset is the global noise addition ratio; clip(A, B, C) is a custom operator with the following operation logic:

[0081] If A < B, then A = B;

[0082] If A > C, then A = C;

[0083] If neither of the above two judgment conditions is met, then A = A.

[0084] On the other hand, the present invention also provides a denoising method for a raw image based on the Bayer format, using the above-mentioned denoising device for a raw image based on the Bayer format, including the following steps:

[0085] S1: The image data input unit obtains the image data of the raw image based on the Bayer format according to the original noisy image;

[0086] S2: The local parameter estimation unit estimates the local denoising intensity and local noise addition ratio corresponding to each pixel point in the raw image according to the image data of the raw image. At the same time, the similarity calculation unit calculates the similarity between different types of neighborhoods in each preset window area of the raw image according to the image data of the raw image;

[0087] S3: The weight calculation unit calculates the weights of each pixel point of the same channel in each preset window area around each pixel point according to the local denoising intensity and the similarity;

[0088] S4: The texture calculation unit calculates the local texture information in the preset window area around each pixel point according to the weights;

[0089] S5: The weight correction unit corrects the weights according to the local texture information to obtain new weights;

[0090] S6: The weighted average unit performs a weighted average on the new weights and each pixel point of the same channel in each preset window area to obtain the denoising result of the current point, thereby obtaining a denoised image; and

[0091] S7: The noise addition unit obtains the final denoising result according to the denoised image and the local noise addition ratio, thereby obtaining the final denoised image.

[0092] Optionally, step S2 specifically includes the following steps:

[0093] S21: the brightness calculation unit performs filtering processing on the same channel pixels of the raw image in each of the preset window areas to obtain a brightness value corresponding to each pixel in the preset window area; wherein the preset window area is a 15×15 area;

[0094] S22: the first piecewise linear mapping unit performs 5-segment linear mapping and square root according to the brightness value corresponding to each pixel point in the preset window area, so as to obtain the local denoising intensity corresponding to each pixel point in the preset window area;

[0095] S23: the second piecewise linear mapping unit performs 5-segment linear mapping according to the brightness value corresponding to each pixel in the preset window area to obtain the local noise return ratio corresponding to each pixel in the preset window area; and

[0096] S24: The similarity calculation unit performs a subtraction operation on corresponding channel pixel points in a 1×2 type neighborhood in 8 directions in each preset window area on the raw image and then calculates the absolute value to obtain 8 similarity matrices.

[0097] Optionally, in S3, the specific implementation method of the weight calculation unit is: 8 weight matrices are obtained by performing a negative exponential function operation on the 8 similarity matrices output by the similarity calculation unit.

[0098] Optionally, in S4, the specific implementation method of the texture calculation unit is: performing high-pass filtering operation on the 8 weight matrices output by the weight calculation unit to obtain 8 texture degrees.

[0099] Optionally, in S5, the specific implementation method of the weight correction unit is: using the 8 texture degrees output by the texture calculation unit as smoothing parameters, performing bilateral filtering on the 8 weight matrices output by the weight calculation unit, and then multiplying the filtering result with each weight matrix to obtain 8 new weight matrices.

[0100] Optionally, in S6, the specific implementation method of the weighted averaging unit is: performing weighted averaging on the corresponding same-channel pixels in each 15×15 area of ​​the raw image output by the image data input unit to obtain 8 denoising results, and finally averaging the 8 results again to obtain a denoised image.

[0101] Optionally, in S7, the specific implementation method of the noise adding back unit is: performing weighted averaging on the denoised image and the local noise adding back ratio to obtain a final result.

[0102] Furthermore, the noise adding back unit performs a weighted averaging operation on the output result of the weighted averaging unit and the image data of the raw image according to the local noise adding back ratio output by the second piecewise linear mapping unit.

[0103] Compared with the prior art, the present invention has the following beneficial effects:

[0104] The present invention provides a denoising device and method for a raw image based on a Bayer format. The denoising device comprises an image data input unit, a local parameter estimation unit, a similarity calculation unit, a weight calculation unit, a texture calculation unit, a weight correction unit, a weighted average unit and a noise back-addition unit. The image data input unit is used to obtain image data of a raw image based on a Bayer format according to an original noisy image; the local parameter estimation unit is used to estimate the local denoising intensity and the local noise back-addition ratio corresponding to each pixel point in the raw image according to the image data of the raw image; the similarity calculation unit is used to calculate the similarity between different types of neighborhoods in each preset window area of ​​the raw image according to the image data of the raw image. Similarity; a weight calculation unit is used to calculate the weight of each pixel point in the same channel in each preset window area around each pixel point according to the local denoising strength and similarity; a texture calculation unit is used to calculate the local texture information in the preset window area around each pixel point according to the weight; a weight correction unit is used to correct the weight according to the local texture information to obtain a new weight; a weighted average unit is used to weighted average the new weight and each pixel point in the same channel in each preset window area to obtain the denoising result of the current point, thereby obtaining a denoised image; and a noise back-addition unit is used to obtain the final denoising result according to the denoised image and the local noise back-addition ratio, thereby obtaining a final denoising image. The present invention calculates the similarity of local neighborhoods of different types through the similarity calculation unit, and the weight self-correction of the weight correction unit can better remove noise. At the same time, the local parameter estimation unit adaptively estimates the denoising parameters (i.e., local denoising strength and local noise back-addition ratio) according to the local information of the raw image, thereby improving the denoising effect and being conducive to hardware implementation. BRIEF DESCRIPTION OF THE DRAWINGS

[0105] Figure 1 A schematic structural diagram of a denoising device for a raw image based on a Bayer format according to an embodiment of the present invention;

[0106] Figure 2 A schematic diagram of the structure of a local parameter estimation unit provided by an embodiment of the present invention;

[0107] Figure 3 Input is the surrounding pixels of the current point of the original image according to an embodiment of the present invention 15×15Schematic diagram of

[0108] Figure 4 A schematic diagram of the structure of a similarity calculation unit provided in one embodiment of the present invention;

[0109] Figure 5 A schematic diagram of eight neighborhood types provided in one embodiment of the present invention;

[0110] Figure 6 A schematic diagram of the structure of a weight correction unit provided by an embodiment of the present invention;

[0111] Figure 7a-7b The figure is a flow chart of a method for denoising a raw image based on a Bayer format according to an embodiment of the present invention.

[0112] Description of reference numerals:

[0113] 1-image data input unit; 2-local parameter estimation unit; 21-brightness calculation unit; 22-first piecewise linear mapping unit; 23-second piecewise linear mapping unit; 3-similarity calculation unit; 311-first neighborhood type; 312-second neighborhood type; 313-third neighborhood type; 314-fourth neighborhood type; 315-fifth neighborhood type; 316-sixth neighborhood type; 317-seventh neighborhood type; 318-eighth neighborhood type; 32-similarity calculation subunit; 4-weight calculation unit; 5-texture calculation unit; 6-weight correction unit; 61-bilateral filtering unit; 7-weighted averaging unit; 8-noise back-addition unit. DETAILED DESCRIPTION

[0114] The following will further describe in detail a denoising device and method for a raw image based on a Bayer format of the present invention. The present invention will be described in more detail below with reference to the accompanying drawings, in which preferred embodiments of the present invention are shown, and it should be understood that those skilled in the art can modify the present invention described herein and still achieve the advantageous effects of the present invention. Therefore, the following description should be understood as being widely known to those skilled in the art and not as a limitation of the present invention.

[0115] For the sake of clarity, not all features of the actual embodiments are described. In the following description, well-known functions and structures are not described in detail because they would clutter the invention with unnecessary detail. It should be recognized that in the development of any actual embodiment, a large number of implementation details must be made to achieve the developer's specific goals, such as changing from one embodiment to another according to the limitations of the relevant system or the relevant business. In addition, it should be recognized that such development work may be complex and time-consuming, but it is just a routine task for those skilled in the art.

[0116] In order to make the purpose and features of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are further described below in conjunction with the accompanying drawings. It should be noted that the accompanying drawings are all in a very simplified form and use inaccurate ratios, which are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention.

[0117] Figure 1 FIG. 1 is a schematic diagram of a structure of a denoising device for a raw image based on a Bayer format according to the present embodiment. Figure 1 As shown, this embodiment provides a denoising device for raw images based on the Bayer format, including an image data input unit 1, a local parameter estimation unit 2, a similarity calculation unit 3, a weight calculation unit 4, a texture calculation unit 5, a weight correction unit 6, a weighted averaging unit 7 and a noise back-addition unit 8.

[0118] The image data input unit 1 is used to obtain image data of a raw image based on the Bayer format according to the original noisy image, and output the image data of the raw image to the local parameter estimation unit 2, the similarity calculation unit 3 and the weighted average unit 7. According to different configurations, the bit width of the image data of the raw image output by the image data input unit 1 is different, and usually defaults to image data with a bit width of 12 bits. The image data includes brightness information.

[0119] Figure 2 A schematic diagram of the structure of a local parameter estimation unit provided in this embodiment. Figure 3 Input is the surrounding pixels of the current point in the original image of this embodiment 15×15 Schematic diagram of Figure 2-3 As shown, the local parameter estimation unit 2 is used to estimate the local denoising strength and the local noise back-addition ratio corresponding to each pixel in the raw image according to the image data of the raw image, and output the local denoising strength to the weight calculation unit 4, and output the local noise back-addition ratio to the noise back-addition unit 8. In detail, the local parameter estimation unit 2 scans the raw image pixel by pixel according to the size of the preset window area to calculate the local denoising strength and the local noise back-addition ratio of each preset window area.

[0120] Since the purpose of denoising intensity estimation is to achieve denoising parameter adaptation, its theoretical basis is the noise model defined in the EMVA1288 standard, see formula (2) for details. That is to say, in theory, the noise intensity can be calibrated according to the average brightness. The purpose of noise back-addition ratio calculation is to calculate the corresponding noise back-addition ratio according to different brightness. Generally speaking, in the same image, the signal-to-noise ratio of the dark area of ​​the image will be lower than that of other areas. Therefore, a certain proportion of the original noisy information is often added.

[0121] Based on the above theory, the local parameter estimation unit 2 can estimate the local noise intensity and the local noise return ratio corresponding to the current pixel point according to the local brightness information of each preset window area of ​​the raw image.

[0122] The local parameter estimation unit 2 includes a brightness calculation unit 21, a first piecewise linear mapping unit 22 and a second piecewise linear mapping unit 23. The brightness calculation unit 21 is used to filter the same-channel pixels of the raw image in each of the preset window areas to obtain the brightness value corresponding to each pixel in the preset window area. In detail, the specific implementation method of the brightness calculation unit 21 calculating the brightness value corresponding to each pixel in the preset window area is: by calculating the weighted average of the same-channel pixels in the preset window area around each pixel in the raw image. In this embodiment, taking the preset window area as a 15x15 area as an example, the brightness calculation unit 21 filters the same-channel pixels of the raw image in each 15×15 area to obtain the brightness value Luma corresponding to each pixel in the preset window area.

[0123] The brightness value Luma satisfies the following calculation formula:

[0124] Luma=(Input 15×15

[16] +Input 15×15

[18] +Input 15×15

[20] +Input 15×15 [twenty two]

[0125] +Input 15×15

[24] +Input 15×15

[26] +Input 15×15

[28] +Input 15×15

[46]

[0126] +Input 15×15

[48] +Input 15×15

[50] +input 15×15

[52] +input 15×15

[54]

[0127] +input 15×15

[56] +input 15×15

[58] +input 15×15

[76] +input 15×15

[78]

[0128] +input 15×15

[80] +input 15×15

[84] +Input 15×15

[86] +Input 15×15

[88]

[0129] +Input 15×15

[106] +Input 15×15

[108] +Input 15×15

[116] +Input 15×15

[118]

[0130] +Input 15×15

[136] +Input 15×15

[138] +Input 15×15

[140] +Input 15×15

[144]

[0131] +Input 15×15

[146] +Input 15×15

[148] +Input 15×15

[166] +Input 15×15

[168]

[0132] +Input 15×15

[170] +Input 15×15

[172] +Input 15×15

[174] +Input 15×15

[176]

[0133] +Input 15×15

[178] +Input 15×15

[196] +Input 15×15

[198] +Input 15×15

[200]

[0134] +input 15×15

[202] +Input 15×15

[204] +Input 15×15

[206] +Input 15×15

[208]

[0135] +2

[0136] ×(Input 15×15

[82] +Input 15×15

[110] +Input 15×15

[114] +Input 15×15

[142] )

[0137] +4×Input 15×15

[112] ) / 56 (3)

[0138] Among them, Input 15×15 The pixel value of the 15x15 area around each pixel.

[0139] The first piecewise linear mapping unit 22 is used to perform 5-segment linear mapping and square root according to the brightness value corresponding to each pixel point in the preset window area to obtain the local denoising intensity corresponding to each pixel point in the preset window area.

[0140] The calculation formula of the estimated value Sigma of the local denoising strength is as follows:

[0141]

[0142] Wherein, Luma is the brightness value of each pixel in the preset window area of ​​the raw image; S s0～4 is the configuration parameter, specifically the slope of each segment in the 5-segment linear mapping; X s0～4 is the configuration parameter, specifically the horizontal coordinate value of each starting point in the 5-segment linear mapping; s0～4 It is a configuration parameter, specifically the vertical coordinate value of the starting point of each segment in the 5-segment linear mapping.

[0143] The second piecewise linear mapping unit 23 is used to perform 5-segment linear mapping according to the brightness value corresponding to each pixel point in the preset window area to obtain the local noise return ratio corresponding to each pixel point in the preset window area.

[0144] The calculation formula of the estimated value AddbackRatio of the local noise add-back ratio is as follows:

[0145]

[0146] Wherein, Luma is the brightness value of each pixel in the preset window area of ​​the raw image; S a0～4 is the configuration parameter, specifically the slope of each segment in the 5-segment linear mapping; X a0～4 is the configuration parameter, specifically the horizontal coordinate value of each starting point in the 5-segment linear mapping; a0～4 It is a configuration parameter, specifically the vertical coordinate value of the starting point of each segment in the 5-segment linear mapping.

[0147] Figure 4 This is a schematic diagram of the structure of the similarity calculation unit provided in this embodiment. Figure 4As shown, the similarity calculation unit 3 is used to calculate the similarity between the neighborhood of each pixel point of the same channel of the raw image in each preset window area and the neighborhood of the central pixel point according to the image data of the raw image. In detail, the specific implementation method of the similarity calculation unit 3 is: the raw image is subjected to a 1×2 type neighborhood in each preset window area (for example, a 15×15 area) in 8 directions. The corresponding channel pixel points are subtracted and then the absolute value operation is calculated to obtain 8 similarity matrices.

[0148] Furthermore, the similarity calculation unit 3 includes 8 neighborhood types arranged in parallel, and a similarity calculation subunit 32. A similarity matrix is ​​calculated through each of the neighborhood types and the similarity calculation subunit 32, and finally 8 similarity matrices are obtained. Figure 5 Schematic diagram of 8 types of neighborhoods provided in this embodiment. Figure 5 As shown, in this embodiment, the eight neighborhood types are, for example, 1×2 neighborhood similarity calculation types in eight directions (specifically, the first neighborhood type 311, the second neighborhood type 312, the third neighborhood type 313, the fourth neighborhood type 314, the fifth neighborhood type 315, the sixth neighborhood type 316, the seventh neighborhood type 317, and the eighth neighborhood type 318). These eight neighborhood types represent the neighborhoods in the vertical and horizontal directions and the diagonal directions of each pixel, respectively, which can better restore the detailed information of the image and effectively reduce the traces of artificial denoising. When calculating the similarity, the similarity calculation subunit 32 needs to subtract the neighborhood of each surrounding pixel (i.e., the 1×2 matrix) from the neighborhood of the central pixel (i.e., the 1×2 matrix) according to the same channel and then calculate the absolute value, and finally obtain 8 7×7 similarity matrices Dist.

[0149] Each similarity matrix Dist i,j,k The calculation formula is as follows:

[0150]

[0151] Among them, Center is the pixel value of each pixel point and its neighborhood in the preset window area of ​​the raw image; Neighbor is the pixel value of other points in the same channel and its neighborhood in the 15×15 area around each pixel point in the preset window area of ​​the raw image; c is the channel, and when c=0, it is the same channel as the channel and the center pixel point, and when c=1, it is a different channel from the center pixel point; Dist is a similarity matrix, and its size is, for example, 7×7×8; i and j are the two-dimensional coordinate positions of the Dist matrix respectively; k is the third-dimensional coordinate position of the Dist matrix.

[0152] The weight calculation unit 4 is used to calculate the weight of each pixel point in the same channel in each preset window area around each pixel point according to the similarity between the neighborhood of each pixel point in the same channel in each preset window area of ​​the raw image and the neighborhood of the central pixel point, and the local denoising intensity corresponding to each pixel point in the raw image, and output the weight to the texture calculation unit 5 and the weight correction unit 6. In detail, the specific implementation method of the weight calculation unit 4 is: 8 weight matrices are obtained by performing negative exponential function operations on the 8 similarity matrices output by the similarity calculation unit 3. Generally speaking, the higher the similarity, the greater the weight, and vice versa.

[0153] Each of the weight matrices Weight1 i,j,k The calculation formula is as follows:

[0154]

[0155] Among them, Dist is a similarity matrix, whose size is, for example, 7×7×8; Sigma is the local denoising intensity corresponding to each pixel point of the raw image in each preset window area; Weight1 is the weight matrix of the same channel points in the 15×15 area around each pixel point, and its size is 7×7×8; e represents the exponential operation; i and j are the two-dimensional coordinate positions of the Dist matrix and the Weight1 matrix respectively; k is the third-dimensional coordinate position of the Dist matrix and the Weight1 matrix.

[0156] The texture calculation unit 5 is used to calculate the local texture information around each pixel in each preset window area according to the weight (specifically 8 weight matrices). The reason why the local area of ​​the original image (i.e., the raw image) is not directly used for texture calculation here is that the original image has not been whitened, the residual values ​​of different brightness areas cannot be unified, and it is more susceptible to noise in low-light scenes. Therefore, the specific implementation method of the texture calculation unit 5 is: the 8 weight matrices output by the weight calculation unit 4 are processed by high-pass filtering to obtain 8 texture degrees.

[0157] Each texture k The calculation formula is as follows:

[0158]

[0159] Among them, Weight is the weight matrix of the points in the same channel in the 15×15 area around each pixel, and its size is, for example, 7×7×8; HighPassKernel is the configuration parameter, specifically the parameter of the high-pass filter; Texture is the texture degree in the 15×15 area around each pixel, and its size is 1×8; i and j are the two-dimensional coordinate positions of the Weight1 matrix respectively; k is the third-dimensional coordinate position of the Weight1 matrix.

[0160] The weight correction unit 6 is used to correct the weight according to the local texture information. Since the weight output by the weight calculation unit 4 only considers the similarity between pixels and does not reflect the directionality of the overall local texture, the weight correction unit 6 performs adaptive bilateral filtering on the weight, wherein the adaptive parameter required for adaptive bilateral filtering is the local texture information. The specific implementation method of the weight correction unit 6 is: using the 8 texture degrees output by the texture calculation unit 5 as smoothing parameters, bilateral filtering is performed on the 8 weight matrices output by the weight calculation unit 4 to obtain the filtering result, and the filtering result is multiplied by each of the weight matrices to obtain 8 new weight matrices.

[0161] Figure 6 This is a schematic diagram of the structure of a weight correction unit provided by an embodiment of the present invention. Figure 6 As shown, the weight correction unit 6 includes a bilateral filtering unit 61, which is used to use the 8 texture degrees output by the texture calculation unit 5 as smoothing parameters to perform bilateral filtering on the 8 weight matrices output by the weight calculation unit 4 to obtain filtering results. The weight correction unit 6 multiplies the filtering results with each weight matrix to obtain 8 new weight matrices, and outputs the new weight matrices to the weighted average unit 7. The formulas involved in the calculation process of the weight correction unit 6 are as follows:

[0162] D i，j，k 2 =(X i，j，k -X center ) 2 +(Y i，j，k -Y center ) 2 (9)

[0163]

[0164] R i，i，k 2 =(Weight i，j，k -Weight1 center，k ) 2 (11)

[0165]

[0166]

[0167] Among them, X i,j is the horizontal coordinate of the point (i, j) in the 7×7 window in each neighborhood type in the Cartesian coordinate system; i,j D is the ordinate of the point (i, j) in the 7×7 window in each neighborhood type in the Cartesian coordinate system; i,j is the Euclidean distance from the point (i, j) in the 7×7 window to the center of the 7×7 window in each neighborhood type; Texture is the texture degree in the 15×15 area around each pixel, which represents the smoothing strength; GainD is a configuration parameter, specifically a fixed gain of 1 for the smoothing strength; GaussianKernel is an intermediate result, specifically the value of the Gaussian kernel, and its size is, for example, 7×7×8; Weight is the weight matrix of the same channel points in the 15×15 area around each pixel, and its size is, for example, 7×7×8; R i,j is the residual of the center point of the point (i, j) in the 7×7 window in each neighborhood type; GainR is a configuration parameter, specifically a fixed gain of 2 for smoothing intensity; ResidualKernel is an intermediate result, specifically the value of the residual kernel, and its size is, for example, 7×7×8; BilateralKernel is the value of the bilateral filter kernel, and its size is, for example, 7×7×8; i and j are the two-dimensional coordinate positions of the Weight1 matrix respectively; k is the third-dimensional coordinate position of the Weight1 matrix.

[0168] After obtaining the bilateral kernel matrix through the above calculation, directly multiply it with the corresponding position of the weight matrix to obtain the corrected weight (i.e., the new weight matrix). Each new weight matrix Weight2 i,j,k The calculation formula is as follows:

[0169]

[0170] Among them, i and j are the two-dimensional coordinate positions of the Weight1 matrix and the Weight2 matrix respectively; k is the third-dimensional coordinate position of the Weight1 matrix and the Weight2 matrix; BilateralKernel is the value of the bilateral filter kernel.

[0171] The weighted averaging unit 7 is used to perform weighted averaging on the weights output by the weight correction unit 6 and each pixel point of the same channel in each preset window area to obtain the denoising result of the pixel point of the current channel of the raw image. The specific implementation method of the weighted averaging unit 7 is to perform weighted averaging on the pixel points of the same channel corresponding to each 15×15 area of ​​the raw image output by the image data input unit 1 to obtain 8 denoising results, and finally average the 8 results again to obtain the denoised image. Furthermore, the weighted averaging unit 7 performs weighted averaging on the pixel point of the current channel in the raw image (such as Figure 3 The black area in the figure) is weighted averaged according to the 8 weights output by the weight correction unit 6, and then the weighted average results of the 8 neighborhood types are averaged.

[0172] The denoising result NrOut satisfies the formula:

[0173]

[0174] Among them, Input 15×15当前点 are the pixels of the same channel in the 15×15 area around each pixel in the raw image (7×7); Weight2 is the weight matrix of the points of the same channel in the 15×15 area around each pixel, and its size is 7×7×8; the size of NrOut is, for example, 1×1; i and j are the two-dimensional coordinate positions of the Dist matrix and the Weight2 matrix respectively; k is the third-dimensional coordinate position of the Dist matrix and the Weight2 matrix.

[0175] The noise adding back unit 8 obtains the final denoising result according to the denoised image and the local noise adding back ratio, thereby completing the denoising of the original noisy image, that is, obtaining the final denoised image. The purpose of the noise adding back unit 8 is to retain a certain amount of noise according to the brightness of the local area on the basis of the denoising completed by the weighted averaging unit 7, mainly for the darker areas in the raw image, so as to improve its signal-to-noise ratio, while avoiding the loss of effective information to a certain extent. In detail, the specific implementation method of the noise adding back unit 8 is: weighted averaging the denoised image and the local noise adding back ratio to obtain the final result. Furthermore, the noise adding back unit 8 performs a weighted averaging operation on the output result of the weighted averaging unit 7 (i.e., the denoised image) and the image data of the raw image according to the local noise adding back ratio output by the second piecewise linear mapping unit 23.

[0176] The final denoising result Output satisfies the following formula:

[0177] W a =clip(AddbackRatio+Offset, 0, 1) (16)

[0178] Output = (1 - W a ) × NrOut + W a × Input(17)

[0179] where AddbackRatio is the local noise add-back ratio corresponding to each pixel point of the raw image in each preset region; W a is an intermediate result, specifically the weight of the noise add-back of the raw image; Offset is a configuration parameter, specifically the global noise add-back ratio; clip(A, B, C) is a custom operator, and the specific operation logic is as follows:

[0180] If A < B, then A = B;

[0181] If A > C, then A = C;

[0182] If neither of the above two judgment conditions is met, then A = A.

[0183] Figure 7a-7b is a schematic flow chart of the denoising method for the raw image based on the Bayer format in this embodiment. As Figure 7a-7b shown, this embodiment also provides a denoising method for the raw image based on the Bayer format, including the following steps:

[0184] S1: The image data input unit 1 obtains the image data of the raw image based on the Bayer format according to the original noisy image;

[0185] S2: The local parameter estimation unit 2 estimates the local denoising intensity and local noise add-back ratio corresponding to each pixel point in the raw image according to the image data of the raw image. At the same time, the similarity calculation unit 3 calculates the similarity between different types of neighborhoods in each preset window region of the raw image according to the image data of the raw image;

[0186] S3: The weight calculation unit 4 calculates the weights of the pixel points of the same channel in each preset window region around each pixel point according to the local denoising intensity and the similarity;

[0187] S4: The texture calculation unit 5 calculates the local texture information in each preset window region around each pixel point according to the weights;

[0188] S5: The weight correction unit 6 corrects the weights according to the local texture information to obtain new weights;

[0189] S6: the weighted averaging unit 7 performs weighted averaging on the new weight and each pixel point of the same channel in each preset window area to obtain a denoising result of the current point, thereby denoising the image;

[0190] S7: The noise adding back unit 8 obtains a final denoising result according to the denoised image and the local noise adding back ratio, thereby obtaining a final denoised image.

[0191] In step S1, according to different configurations, the image data input unit 1 outputs image data of the raw image with different bit widths, usually 12-bit image data by default.

[0192] Step S2 specifically includes the following steps:

[0193] S21: The brightness calculation unit 21 performs filtering processing on the same channel pixels of the raw image in each of the preset window areas to obtain the brightness value corresponding to each pixel in the preset window area. The preset window area is, for example, a 15×15 area.

[0194] S22: The first piecewise linear mapping unit 22 performs 5-segment linear mapping and square root according to the brightness value corresponding to each pixel point in the preset window area to obtain the local denoising intensity corresponding to each pixel point in the preset window area.

[0195] S23: The second piecewise linear mapping unit 23 performs 5-segment linear mapping according to the brightness value corresponding to each pixel point in the preset window area to obtain the local noise return ratio corresponding to each pixel point in the preset window area.

[0196] S24: the similarity calculation unit 3 performs a subtraction operation on corresponding channel pixel points in a 1×2 type neighborhood in 8 directions in each preset window area on the raw image and then calculates the absolute value to obtain 8 similarity matrices.

[0197] In S3, the specific implementation method of the weight calculation unit 4 is: 8 weight matrices are obtained by performing negative exponential function operation on the 8 similarity matrices output by the similarity calculation unit 3.

[0198] In S4, the specific implementation method of the texture calculation unit 5 is: the 8 weight matrices output by the weight calculation unit 4 are processed by high-pass filtering to obtain 8 texture degrees.

[0199] In S5, the specific implementation method of the weight correction unit 6 is: using the 8 texture degrees output by the texture calculation unit 5 as smoothing parameters, bilateral filtering is performed on the 8 weight matrices output by the weight calculation unit 4, and then the filtering result is multiplied by each weight matrix to obtain 8 new weight matrices.

[0200] In S6, the specific implementation method of the weighted averaging unit 7 is: weighted averaging the corresponding same-channel pixels in each 15×15 area of ​​the raw image output by the image data input unit 1 to obtain 8 denoising results, and finally averaging the 8 results again to obtain a denoised image.

[0201] In S7, the specific implementation method of the noise adding unit 8 is: weighted average the denoised image and the local noise adding ratio to obtain a final result. Further, the noise adding unit 8 performs a weighted average operation on the output result of the weighted averaging unit 7 and the image data of the raw image according to the local noise adding ratio output by the second piecewise linear mapping unit 23.

[0202] In summary, the present invention provides a denoising device and method for a raw image based on a Bayer format, which can better remove noise by calculating the similarity of local neighborhoods of different types and a weight self-correction method. At the same time, according to the local information of the raw image, the denoising parameters are adaptively estimated, which improves the denoising effect of the traditional method and is conducive to hardware implementation.

[0203] In addition, it should be noted that, unless otherwise specified or indicated, the terms "first" and "second" in the specification are only used to distinguish the various components, elements, steps, etc. in the specification, and are not used to indicate the logical relationship or sequential relationship between the various components, elements, steps, etc.

[0204] It is to be understood that, although the present invention has been disclosed as a preferred embodiment, the above embodiment is not intended to limit the present invention. For any person skilled in the art, without departing from the scope of the technical solution of the present invention, the technical content disclosed above can be used to make many possible changes and modifications to the technical solution of the present invention, or modified into equivalent embodiments of equivalent changes. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the scope of protection of the technical solution of the present invention.

Claims

1. A denoising device for a raw image based on a Bayer format, characterized in that: include: An image data input unit, used for obtaining image data of a raw image based on a Bayer format according to an original noisy image; A local parameter estimation unit, used for estimating the local denoising intensity and the local noise adding back ratio corresponding to each pixel point in the raw image according to the image data of the raw image; A similarity calculation unit is used to calculate the similarity between 8 different types of neighborhoods of each pixel point in the same channel of the raw image in each preset window area and 8 different types of neighborhoods of the central pixel point according to the image data of the raw image, wherein the 8 neighborhood types are the neighborhoods of each pixel point in the vertical and horizontal directions and the neighborhoods in the diagonal direction thereof; A weight calculation unit, used for calculating the weight of each pixel point in the same channel in each preset window area around each pixel point according to the local denoising strength and similarity; A texture calculation unit, used to calculate local texture information in the preset window area around each pixel point according to the weight; A weight correction unit, used for correcting the weight according to the local texture information to obtain a new weight; A weighted averaging unit, configured to perform weighted averaging on the new weight and each pixel point of the same channel in each preset window area to obtain a denoising result of the current point, thereby obtaining a denoised image; and The noise adding back unit is used to obtain a final denoising result according to the denoised image and the local noise adding back ratio, so as to obtain a final denoised image.

2. The denoising device for raw images based on the Bayer format as claimed in claim 1, characterized in that: The preset window area is a 15x15 area.

3. The denoising device for raw images based on the Bayer format as claimed in claim 2, characterized in that: The local parameter estimation unit comprises: A brightness calculation unit, used for filtering the same-channel pixels of the raw image in each of the preset window areas to obtain a brightness value corresponding to each pixel in the preset window area; A first piecewise linear mapping unit, configured to perform 5-segment linear mapping and square root according to the brightness value corresponding to each pixel point in the preset window area, so as to obtain a local denoising intensity corresponding to each pixel point in the preset window area; and The second piecewise linear mapping unit is used to perform 5-segment linear mapping according to the brightness value corresponding to each pixel point in the preset window area to obtain the local noise return ratio corresponding to each pixel point in the preset window area.

4. The denoising device for raw images based on the Bayer format as claimed in claim 3, characterized in that: The brightness value Luma satisfies the following calculation formula: Luma=(Input 15×15 [16]+Input 15×15 [18]+Input 15×15 [20]+Input 15×15 [22]+Input 15×15 [24]+Input 15×15 [26]+Input 15×15 [28]+Input 15×15 [46]+Input 15×15 [48]+Input 15×15 [50]+Input 15×15 [52]+Input 15×15 [54]+Input 15×15 [56]+Input 15×15 [58]+Input 15×15 [76]+Input 15×15 [78]+Input 15×15 [80]+Input 15×15 [84]+Input 15×15 [86]+Input 15×15 [88]+Input 15×15 [106]+Input 15×15 [108]+Input 15×15 [116]+Input 15×15 [118]+Input 15×15 [136]+Input 15×15 [138]+Input 15×15 [140]+Input 15×15 [144]+Input 15×15 [146]+Input 15×15 [148]+Input 15×15 [166]+Input 15×15 [168]+Input 15×15 [170]+Input 15×15 [172]+Input 15×15 [174]+Input 15×15 [176]+Input 15×15 [178]+Input 15×15 [196]+Input 15×15 [198]+Input 15×15 [200]+Input 15×15 [202]+Input 15×15 [204]+Input 15×15 [206]+Input 15×15 [208]+2×(Input 15×15 [82]+Input 15×15 [110]+Input 15×15 [114]+Input 15×15 [142])+4×Input 15×15 [112]) / 56 Among them, Input 15×15 The pixel value of the 15x15 area around each pixel.

5. The denoising device for raw images based on the Bayer format as claimed in claim 4, characterized in that: The calculation formula of the estimated value Sigma of the local denoising strength is as follows: Wherein, Luma is the brightness value of each pixel in the preset window area of ​​the raw image; S s0～4 is the slope of each of the 5 linear mappings; 0, X s1～4 is the horizontal coordinate value of each starting point in the 5-segment linear mapping; X s5 Y is the horizontal coordinate value of the last endpoint of the 5-segment linear mapping; s0～4 It is the ordinate value of the starting point of each segment in the 5-segment linear mapping.

6. The denoising device for raw images based on the Bayer format as claimed in claim 4, characterized in that: The calculation formula of the estimated value AddbackRatio of the local noise add-back ratio is as follows: Wherein, Luma is the brightness value of each pixel in the preset window area of ​​the raw image; S a0～4 is the slope of each of the 5 linear mappings; 0, X a1～4 is the horizontal coordinate value of each starting point in the 5-segment linear mapping; X a5 Y is the horizontal coordinate value of the last endpoint of the 5-segment linear mapping; a0～4 It is the ordinate value of the starting point of each segment in the 5-segment linear mapping.

7. The denoising device for raw images based on the Bayer format as claimed in claim 2, characterized in that: The similarity calculation unit includes 8 neighborhood types arranged in parallel and a similarity calculation subunit. The similarity calculation unit calculates a similarity matrix through each of the neighborhood types and the similarity calculation subunit, and finally obtains 8 similarity matrices.

8. The denoising device for raw images based on the Bayer format as claimed in claim 7, characterized in that: Each similarity matrix Dist i,j,k The calculation formula is as follows: Among them, Center is the pixel value of each pixel point and its neighborhood in the preset window area of the raw image; Neighbor is the pixel value of other points of the same channel and their neighborhoods within a 15×15 area around each pixel point in the preset window area of the raw image; c is the channel, and c = 0 represents the same channel as the central pixel point, c = 1 represents a channel different from the central pixel point; Dist is the similarity matrix, with a size of 7×7×8; i and j are the two-dimensional coordinate positions of the Dist matrix respectively; k is the three-dimensional coordinate position of the Dist matrix.

9. The denoising device for raw images based on the Bayer format as claimed in claim 7, characterized in that: The weight calculation unit obtains 8 weight matrices according to the 8 similarity matrices, each of which has a weight matrix Weight1 i,j,k The calculation formula is as follows: Among them, Dist is the similarity matrix, with a size of 7×7×8; Sigma is the local denoising intensity corresponding to each pixel point in each preset window area of the raw image; Weight1 is the weight matrix of points of the same channel within a 15×15 area around each pixel point, with a size of 7×7×8; e represents the exponential operation; i and j are the two-dimensional coordinate positions of the Dist matrix and the Weight1 matrix respectively; k is the three-dimensional coordinate position of the Dist matrix and the Weight1 matrix.

10. The denoising device for raw images based on the Bayer format as claimed in claim 9, characterized in that: The texture calculation unit obtains 8 textures according to the 8 weight matrices, each of which has a texture k The calculation formula is as follows: Among them, Weight1 is the weight matrix of points of the same channel within a 15×15 area around each pixel point, with a size of 7×7×8; HighPassKernel is the parameter of the high-pass filter; Texture is the texture degree within a 15×15 area around each pixel point, with a size of 1×8; i and j are the two-dimensional coordinate positions of the Weight1 matrix respectively; k is the three-dimensional coordinate position of the Weight1 matrix.

11. The denoising device for raw images based on the Bayer format as claimed in claim 10, characterized in that: The weight correction unit includes a bilateral filtering unit, and the bilateral filtering unit is used to perform bilateral filtering on the 8 weight matrices output by the weight calculation unit by using the 8 texture degrees output by the texture calculation unit as smoothing parameters to obtain 8 new weight matrices.

12. The denoising device for raw images based on the Bayer format as claimed in claim 11, characterized in that: Each new weight matrix Weight2 i,j,k The following calculation formula is satisfied: D i,j,k 2 =(X i,j,k -X center ) 2 +(Y i,j,k -Y center ) 2 R i,j,k 2 =(Weight1 i,j,k -Weight1 center,k ) 2 Among them, X i,j is the horizontal coordinate of the point (i, j) in the 7×7 window in each neighborhood type in the Cartesian coordinate system; i,j D is the ordinate of the point (i, j) in the 7×7 window in each neighborhood type in the Cartesian coordinate system; i,j is the Euclidean distance from the point (i, j) in the 7×7 window to the center of the 7×7 window in each neighborhood type; Texture is the texture degree in the 15×15 area around each pixel, which represents the smoothing strength; GainD is the fixed gain 1 of the smoothing strength; GaussianKernel is the value of the Gaussian kernel, whose size is 7×7×8; Weight is the weight matrix of the points in the same channel in the 15×15 area around each pixel, whose size is 7×7×8; R i,j is the residual of the center point of the point (i, j) in the 7×7 window in each neighborhood type; GainR is the fixed gain of smoothing intensity 2; ResidualKernel is the value of the residual kernel, and its size is 7×7×8; BilateralKernel is the value of the bilateral filter kernel, and its size is 7×7×8; i and j are the two-dimensional coordinate positions of the Weight1 matrix and the Weight2 matrix respectively; k is the third-dimensional coordinate position of the Weight1 matrix and the Weight2 matrix.

13. The denoising device for raw images based on the Bayer format as claimed in claim 12, characterized in that: The denoising result NrOut obtained by the weighted average unit satisfies the formula: Among them, Input 15×15当前点 are the pixels of the same channel in the 15×15 area around each pixel in the raw image (7×7); Weight2 is the weight matrix of the points of the same channel in the 15×15 area around each pixel, and its size is 7×7×8; the size of NrOut is 1×1; i and j are the two-dimensional coordinate positions of the Dist matrix and the Weight2 matrix respectively; k is the third-dimensional coordinate position of the Dist matrix and the Weight2 matrix.

14. The denoising device for raw images based on the Bayer format as claimed in claim 13, characterized in that: The final denoising result Output obtained by the noise addition unit satisfies the following formula: W a =clip(AddbackRatio+Offset,0,1) Output=(1-W a )×NrOut+W a ×Input Wherein, AddbackRatio is the local noise adding ratio corresponding to each pixel point in each preset area of ​​the raw image; a is the weight of the noise added back to the raw image; Offset is the global noise added back ratio; clip(A,B,C) is a custom operator with the following operation logic: If A < B, then A = B; If A > C, then A = C; If neither of the above two judgment conditions is met, then A = A.

15. A method for denoising a raw image based on a Bayer format, using the device for denoising a raw image based on a Bayer format as claimed in any one of claims 1 to 14, characterized in that: It includes the following steps: S1: The image data input unit obtains the image data of the raw image based on the Bayer format according to the original noisy image; S2: The local parameter estimation unit estimates the local denoising intensity and the local noise addition ratio corresponding to each pixel point in the raw image according to the image data of the raw image. Meanwhile, the similarity calculation unit calculates the similarity between different types of neighborhoods within each preset window area of the raw image according to the image data of the raw image; S3: The weight calculation unit calculates the weights of each pixel point of the same channel within each preset window area around each pixel point according to the local denoising intensity and the similarity; S4: The texture calculation unit calculates the local texture information within the preset window area around each pixel point according to the weights; S5: the weight correction unit corrects the weight according to the local texture information to obtain a new weight; S6: the weighted averaging unit performs weighted averaging on the new weight and each pixel point of the same channel in each preset window area to obtain a denoising result of the current point, thereby obtaining a denoised image; as well as S7: The noise adding back unit obtains a final denoising result according to the denoised image and the local noise adding back ratio, thereby obtaining a final denoised image.

16. The method for denoising a raw image based on the Bayer format as claimed in claim 15, characterized in that: Step S2 specifically includes the following steps: S21: the brightness calculation unit performs filtering processing on the same channel pixels of the raw image in each of the preset window areas to obtain a brightness value corresponding to each pixel in the preset window area; wherein the preset window area is a 15×15 area; S22: the first piecewise linear mapping unit performs 5-segment linear mapping and square root according to the brightness value corresponding to each pixel point in the preset window area, so as to obtain the local denoising intensity corresponding to each pixel point in the preset window area; S23: the second piecewise linear mapping unit performs 5-segment linear mapping according to the brightness value corresponding to each pixel in the preset window area to obtain the local noise return ratio corresponding to each pixel in the preset window area; and S24: The similarity calculation unit performs a subtraction operation on corresponding channel pixel points in a 1×2 type neighborhood in 8 directions in each preset window area on the raw image and then calculates the absolute value to obtain 8 similarity matrices.

17. The method for denoising a raw image based on a Bayer format as claimed in claim 15, characterized in that: In S3, the specific implementation method of the weight calculation unit is: 8 weight matrices are obtained by performing negative exponential function operation on the 8 similarity matrices output by the similarity calculation unit.

18. The method for denoising a raw image based on the Bayer format as claimed in claim 15, characterized in that: In S4, the specific implementation method of the texture calculation unit is: the 8 weight matrices output by the weight calculation unit are processed by high-pass filtering to obtain 8 texture degrees.

19. The method for denoising a raw image based on the Bayer format as claimed in claim 15, characterized in that: In S5, the specific implementation method of the weight correction unit is: using the 8 texture degrees output by the texture calculation unit as smoothing parameters, performing bilateral filtering on the 8 weight matrices output by the weight calculation unit, and then multiplying the filtering result with each weight matrix to obtain 8 new weight matrices.

20. The method for denoising a raw image based on the Bayer format as claimed in claim 15, characterized in that: In S6, the specific implementation method of the weighted averaging unit is: weighted averaging the corresponding same-channel pixels in each 15×15 area of ​​the raw image output by the image data input unit to obtain 8 denoising results, and finally averaging the 8 results again to obtain a denoised image.

21. The method for denoising a raw image based on a Bayer format as claimed in claim 15, characterized in that: In S7, the specific implementation method of the noise adding back unit is: weighted average the denoised image and the local noise adding back ratio to obtain a final result.

22. The method for denoising a raw image based on the Bayer format as claimed in claim 21, characterized in that: The noise adding unit performs a weighted averaging operation on the output result of the weighted averaging unit and the image data of the raw image according to the local noise adding ratio output by the second piecewise linear mapping unit.

Images