Bayer image interpolation method based on brightness map guidance

Abstract

The invention provides a Bayer image interpolation method based on brightness map guidance. The Bayer image interpolation method comprises the steps of S1, brightness estimation; s2, noise calibration: utilizing a camera to aim at the gray scale card for shooting under corresponding conditions to obtain raw data, and counting the average value and the standard deviation of each gray scale block to obtain a noise calibration result sigma (L); l represents a brightness value and is obtained by estimating an average value of gray scale blocks, and sigma (L) represents a noise standard deviation calculated under the brightness; s3, noise elimination: performing filtering processing on the brightness image by using a noise calibration result; and S4, guiding interpolation. The method depends on a noise calibration result, is high in noise robustness, and can adapt to various complex practical application scenes; the method is low in complexity, interpolation results in all directions do not need to be calculated in a weighted mode, and the algorithm complexity is greatly reduced; and the imaging effect is good, and the better imaging effect can be obtained by using guided filtering for interpolation.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a Bayer image interpolation method guided by a brightness map. Background Art

[0002] In the prior art, the purpose of Bayer image interpolation is to interpolate the Bayer image into a complete color image. The original bilinear interpolation algorithm has problems such as zipper effect, low clarity, and serious pseudo-color. Therefore, scholars have invented more advanced interpolation methods. Currently, the most mainstream Bayer image interpolation methods mainly include color difference method and residual method.

[0003] Color difference method. A basic assumption is that the hue and brightness of an RGB image change slowly within a local range, which is manifested as the constancy of local color difference. The constancy of color difference can be used to interpolate the complete G channel information, and then the complete R channel and B channel information can be interpolated based on the complete G channel information and color difference information to obtain a complete RGB image. The color difference method usually needs to be completed in 6 steps:

[0004] Step 1: Use linear interpolation to obtain the horizontal interpolation results and vertical interpolation results of the R, G, and B channels respectively;

[0005] Step 2: Use the interpolation result of the previous step to calculate the horizontal interpolation color difference RG and BG, and at the same time calculate the vertical interpolation color difference RG and BG;

[0006] Step 3: Calculate the color difference weights in the horizontal and vertical directions;

[0007] Step 4: Calculate the final color difference estimation values ​​RG and BG using the color difference weights and color difference values ​​in the two directions;

[0008] Step 5: Superimpose the final color difference estimation value on the R channel and B channel of the original Bayer image to restore the complete G channel information;

[0009] Step 6: Use the principle of constant color difference to restore the R and B channel information.

[0010] Residual method. The comprehensive effect of the residual method is better than that of the color difference method. It assumes that the three channels of the image satisfy linear correlation in the local range, and uses the guided upsampling method to complete the image interpolation. The basic steps of the residual method are the same as those of the color difference method, except that: in the first step, guided upsampling is used to complete the horizontal and vertical interpolation; in the sixth step, guided upsampling is used to complete the interpolation of the R and B channels.

[0011] However, the main defects of the prior art are:

[0012] 1. The complexity and effect of existing methods are contradictory. Although the mainstream color difference method and residual method have good interpolation effects, their complexity is too high. The bilinear interpolation method has low complexity but poor imaging effect.

[0013] 2. Poor robustness. Existing methods are sensitive to noise. Once noise appears in the image, pseudo-texture / pseudo-color and other phenomena are easily produced in the generated image.

[0014] In addition, common terms in the prior art include:

[0015] 1. Bayer image: Most color cameras use 3D photoelectric sensors to obtain image information. A color filter array (CFA) is covered on the sensor surface. The most common CFA format is the Bayer format. The raw data collected by the Bayer format sensor is called a Bayer image. Figure 1 The figure shows a common Bayer image diagram.

[0016] 2. Bayer image interpolation: The process of restoring a bayer image to a complete RGB image.

[0017] 3. Resolution: The ability to distinguish details of the photographed object. The higher the resolution, the richer the image details and the higher the clarity.

[0018] 4. Pseudo color: When interpolating a Bayer image, if the color information is interpolated incorrectly, abnormal colors (False Color) will easily appear, which is called pseudo color.

[0019] 5. Mosaic phenomenon: When interpolating Bayer images, if the brightness information is interpolated incorrectly, abnormal textures are likely to appear. The intuitive manifestation in the image is the checkerboard phenomenon or zipper effect, which can be collectively referred to as the mosaic phenomenon. Summary of the invention

[0020] In order to solve the above problems, the purpose of the present application is: This method uses a noise elimination method based on noise calibration and a guided interpolation method based on grayscale images to solve the problems existing in the prior art.

[0021] Specifically, the present invention provides a Bayer image interpolation method based on brightness map guidance, the method comprising the following steps:

[0022] S1. Brightness estimation:

[0023] The brightness estimation result of the Bayer image is calculated using the following formula:

[0024]

[0025] Where (i, j) represents the coordinates of the point to be estimated; L(·) represents the brightness estimation result; I(·) represents the pixel value of the corresponding point in the Bayer image;

[0026] S2. Noise calibration:

[0027] Under the corresponding conditions, use the camera to shoot at the grayscale card to obtain raw data, and then calculate the average value and standard deviation of each grayscale block to obtain the noise calibration result σ(L); L represents the brightness value, which is estimated by the average value of the grayscale block, and σ(L) represents the noise standard deviation calculated at this brightness;

[0028] S3. Noise cancellation:

[0029] Use the noise calibration result to filter the brightness image:

[0030]

[0031] Where L′(·) represents the result after Gaussian filtering, L(·) represents the brightness image obtained in step 1, M and N represent the radius of the filter window, and H g (·) represents the Gaussian function, which is calculated as:

[0032]

[0033] where k is the overall filter strength parameter specified by the user and σ′(·) is the noise standard deviation obtained by noise calibration:

[0034] σ′(i,j)=σ(L(i,j))

[0035] S4. Guided interpolation

[0036] Using the denoised brightness image in step S3 as the guide image, the interpolation of the Bayer image is completed:

[0037]

[0038]

[0039]

[0040] in, Respectively represent the estimation results of the three channels R, G, and B at the corresponding pixel position (i, j); are the fitting coefficients estimated by the least squares method:

[0041]

[0042]

[0043]

[0044] Where r is the radius of the local fitting window; M R (·), M G (·), M B (·) is a mask image used to mark the location of valid samples: when the current point in the Bayer image is R (or G, B), M R (·)(or M G (·), M B (·)) takes 1 otherwise it takes 0.

[0045] In step S1, it is assumed that a 9*9 template centered on the R pixel is shown in the following table, and the 2*2 thick frame window in the center represents the data range involved in the brightness estimation of the center point:

[0046]

[0047] The step S2 further comprises:

[0048] (1) Under the corresponding conditions, use the camera to shoot the 24-color card to obtain the Bayer image, and calculate the average value and standard deviation of each color block (L i ,σ i ), where the subscript i represents the color block number, L i and σ i Respectively represent the mean and standard deviation of the corresponding color blocks;

[0049] (2) The noise calibration result σ(L) is obtained by using the least squares method, where L represents the brightness value and σ(L) represents the noise standard deviation calculated at the brightness. The least squares fitting formula is: The fitting coefficients are:

[0050]

[0051]

[0052] In step S3, assuming a 3*3 filter window, its corresponding 9*9 template is shown as follows, and the central 3*3 thick frame window represents the data range involved in the denoising operation of the central point:

[0053]

[0054] In step S4, assuming that the fitting window is 7*7 and the R pixel is the center, the channels that need to be interpolated are G and B. Taking the 9*9 template as an example, the thick frame window in the center represents the data range involved in the center point guided interpolation. The schematic table of each image in this step is as follows:

[0055]

[0056]

[0057] Therefore, the advantages of this application are:

[0058] 1. Relying on the results of noise calibration, it has high robustness to noise and can adapt to various complex practical application scenarios;

[0059] 2. The method has low complexity and does not require weighted calculation of interpolation results in all directions, which greatly reduces the complexity of the algorithm;

[0060] 3. The imaging effect is better. By using guided filtering for interpolation, better imaging effect can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0062] Figure 1 This is a common Bayer image diagram.

[0063] Figure 2 It is a flow chart of the present application method. DETAILED DESCRIPTION

[0064] In order to more clearly understand the technical content and advantages of the present invention, the present invention is now further described in detail in conjunction with the accompanying drawings.

[0065] like Figure 2 As shown, the method relates to a Bayer image interpolation method based on brightness map guidance, and the specific method of the invention is divided into the following steps:

[0066] S1. Brightness estimation:

[0067] The brightness estimation result of the Bayer image is calculated using the following formula:

[0068]

[0069] Where (i, j) represents the coordinates of the point to be estimated; L(·) represents the brightness estimation result; I(·) represents the pixel value of the corresponding point in the Bayer image; assuming that the 9*9 template centered on the R pixel is shown in the following table, the 2*2 thick frame window in the center represents the data range involved in the brightness estimation of the center point:

[0070]

[0071] S2. Noise calibration:

[0072] (1) Under the corresponding conditions, use the camera to shoot the 24-color card to obtain the Bayer image, and calculate the average value and standard deviation of each color block (L i ,σ i ), where the subscript i represents the color block number, L i and σ i Respectively represent the mean and standard deviation of the corresponding color blocks;

[0073] (2) The noise calibration result σ(L) is obtained by using the least squares method, where L represents the brightness value and σ(L) represents the noise standard deviation calculated at the brightness. The least squares fitting formula is: The fitting coefficients are:

[0074]

[0075]

[0076] S3. Noise cancellation:

[0077] Use the noise calibration result to filter the brightness image:

[0078]

[0079] Where L′(·) represents the result after Gaussian filtering, L(·) represents the brightness image obtained in step 1, M and N represent the radius of the filter window, and H g (·) represents the Gaussian function, which is calculated as:

[0080]

[0081] where k is the overall filter strength parameter specified by the user and σ′(·) is the noise standard deviation obtained by noise calibration:

[0082] σ′(i,j)=σ(L(i,j))

[0083] Taking the 3*3 filter window as an example, the corresponding 9*9 template is shown in the following table. The 3*3 thick frame window in the center indicates the data range involved in the denoising operation of the center point:

[0084]

[0085] S4. Guided interpolation

[0086] Using the denoised brightness image in step S3 as the guide image, the interpolation of the Bayer image is completed:

[0087]

[0088]

[0089]

[0090] in, Respectively represent the estimation results of the three channels R, G, and B at the corresponding pixel position (i, j); are the fitting coefficients estimated by the least squares method:

[0091]

[0092]

[0093]

[0094] Where r is the radius of the local fitting window; M R (·), M G (·), M B (·) is a mask image used to mark the location of valid samples: when the current point in the Bayer image is R (or G, B), M R (·)(or M G (·), M B (·)) takes 1 otherwise it takes 0.

[0095] Assuming the fitting window is 7*7 and the R pixel is the center, the channels that need to be interpolated are G and B. Taking the 9*9 template as an example, the thick frame window in the center represents the data range involved in the center point guided interpolation. The schematic table of each image in this step is as follows:

[0096]

[0097]

[0098] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A Bayer image interpolation method based on brightness map guidance, characterized in that: The method comprises the following steps: S1. Brightness estimation: The brightness estimation result of the Bayer image is calculated using the following formula: Where (i, j) represents the coordinates of the point to be estimated; L(·) represents the brightness estimation result; I(·) represents the pixel value of the corresponding point in the Bayer image; S2. Noise calibration: Under the corresponding conditions, use the camera to shoot at the grayscale card to obtain raw data, and then calculate the average value and standard deviation of each grayscale block to obtain the noise calibration result σ(L); L represents the brightness value, which is estimated by the average value of the grayscale block, and σ(L) represents the noise standard deviation calculated at this brightness; S3. Noise cancellation: Use the noise calibration result to filter the brightness image: Where L′(·) represents the result after Gaussian filtering, L(·) represents the brightness image obtained in step 1, M and N represent the radius of the filter window, and H g (·) represents the Gaussian function, which is calculated as: where k is the overall filter strength parameter specified by the user and σ′(·) is the noise standard deviation obtained by noise calibration: σ′(i,j)=σ(L(i,j)) S4. Guided interpolation Using the denoised brightness image in step S3 as the guide image, the interpolation of the Bayer image is completed: in, Respectively represent the estimation results of the three channels R, G, and B at the corresponding pixel position (i, j); are the fitting coefficients estimated by the least squares method: Where r is the radius of the local fitting window; M R (·), M G (·), M B (·) is a mask image used to mark the location of valid samples: when the current point in the Bayer image is R or G, B, M R (·) or M G (·), M B (·) takes 1 otherwise it takes 0.

2. The Bayer image interpolation method based on brightness map guidance according to claim 1, characterized in that: In step S1, it is assumed that a 9*9 template centered on the R pixel is shown in the following table, and the 2*2 thick-line frame window in the center represents the data range involved in the brightness estimation of the center point:

3. The Bayer image interpolation method based on brightness map guidance according to claim 2, characterized in that: The step S2 further comprises: (1) Under the corresponding conditions, use the camera to shoot the 24-color card to obtain the Bayer image, and calculate the average value and standard deviation of each color block (L i ,σ i ), where the subscript i represents the color block number, L i and σ i Respectively represent the mean and standard deviation of the corresponding color blocks; (2) The noise calibration result σ(L) is obtained by using the least squares method, where L represents the brightness value and σ(L) represents the noise standard deviation calculated at the brightness. The least squares fitting formula is: The fitting coefficients are:

4. The Bayer image interpolation method based on brightness map guidance according to claim 3, characterized in that: In step S3, assuming a 3*3 filter window, its corresponding 9*9 template is shown as follows, and the 3*3 thick frame window in the center represents the data range involved in the denoising operation of the center point:

5. The Bayer image interpolation method based on brightness map guidance according to claim 4, characterized in that: In step S4, assuming that the fitting window is 7*7 and the R pixel is the center, the channels that need to be interpolated are G and B. Taking the 9*9 template as an example, the thick frame window in the center represents the data range involved in the center point guided interpolation. The schematic table of each image in this step is as follows:

Images