RGBW array color reconstruction method based on guided filter interpolation

Abstract

The application discloses an RGBW array color reconstruction method based on guided filter interpolation, and belongs to the fields of color imaging and image processing. The application realizes the method as follows: based on the color ratio law and the characteristics of guided filter in keeping image detail area, the gray correlation of the brightness channel W information and the RGB color image information is mined, the residual RI is combined with the minimum-Laplacian residual MLRI, and the appropriate iteration number is adaptively selected on each pixel for iteration; aiming at the spatial characteristics of the RGBW filter array, based on the smoothness of the residual, a residual interpolation algorithm with gradually improved sampling rate and pixel-by-pixel iteration is designed; the RGBW array color reconstruction is realized on the RGBW CFA mosaic image based on the guided filter interpolation, clear detail information can be restored, color aliasing and jaggy effect can be reduced, the RGB three-channel color image is obtained, the reconstructed image is more in line with the observation requirements of the human eye, and natural color imaging is realized.

Description

Technical Field

[0001] This invention relates to a color reconstruction method for an RGBW array based on guided filter interpolation, belonging to the field of color imaging and image processing technology. Background Technology

[0002] Currently, most single-chip color digital cameras use Bayer color filter arrays (CFAs) to capture color spectral information, limiting the spectrum to the visible light band. Color reconstruction is achieved by calculating the full-band information of each pixel through a demosaicing algorithm. To fully utilize the rich information in the light source spectrum, one G channel of the Bayer CFA is replaced with a W channel, resulting in the RGBW CFA. The two CFA arrangements are as follows... Figure 1 As shown. Therefore, researchers have proposed several color reconstruction methods based on novel RGBW color filter arrays (CFAs). Similar to RGGB Bayer filter arrays, the RGBW channels of RGBW filter arrays are also spatially discrete, thus requiring interpolation reconstruction of each channel's image. Different interpolation reconstruction methods result in varying degrees of residual mosaic images and color aliasing, directly affecting the color imaging quality in daylight and low light conditions.

[0003] In recent years, color reconstruction methods for novel RGBW filter arrays have become an important research direction. Losson et al. proposed a standard reconstruction method for RGBW filter arrays, which involves converting color pixels into a Bayer array, reconstructing the luminance image using a traditional color reconstruction algorithm, performing downsampling, synthesizing the difference image with the color image, interpolating back to the original resolution, and then combining it with the luminance image to obtain the final image. Mabuchi et al. proposed converting the RGBW filter array into a Bayer array arrangement before implementing a traditional color reconstruction algorithm. Rafinazari et al. proposed a frequency domain-based method that effectively reduces false color phenomena through luminance-chrominance demultiplexing. Paul et al. proposed a shading-based method that can reconstruct colors using a small number of color sample points. When applied to video reconstruction, this method can obtain images with a high signal-to-noise ratio. Kim et al. proposed a color difference-based method, which uses gradient detection to detect color differences between channels and enhances high-frequency information in diagonal regions; this method is an extension of traditional color reconstruction algorithms. Kang et al. proposed a pixel block nonlocal regularization-based method that considers the correlation between channels. Kwan et al. proposed a brightness-guided resolution enhancement method, which combines an interpolated high-resolution brightness image with a low-resolution color image and reconstructs the color image using various enhancement methods. Kwan et al. subsequently improved the brightness-guided resolution enhancement method by introducing the Demonet depth model and two feedback structures, and validated it on low-light scene images.

[0004] Because the luminance W channel of the filter has a wide spectral transmission band, it theoretically has a strong ability to preserve image details, especially with a high signal-to-noise ratio under low-light conditions. However, most current RGBW array color reconstruction methods simply process the color image and the luminance W image separately, and then design reconstruction algorithms based on the characteristics of the Bayer array, without fully utilizing the advantages of the luminance W channel information. Color difference-based methods have some disadvantages in RGBW color reconstruction. When measuring the energy difference between the reconstructed pixel values ​​and the initial filter array pixel values, the smoothness of color difference is weaker than that of residuals. In traditional Bayer array color reconstruction research, residual interpolation, iterative residual interpolation, Laplacian residual interpolation, adaptive residual interpolation, and other residual interpolation-based methods all have better peak signal-to-noise ratios than color difference-based methods. Summary of the Invention

[0005] To address the problem that most current RGBW array color reconstruction methods do not fully utilize the luminance W channel information and result in insufficient final reconstruction imaging accuracy, the present invention aims to provide an RGBW array color reconstruction method based on guided filter interpolation. This method leverages the color ratio law and the characteristic of guided filtering to preserve image detail regions, and mines the grayscale correlation between luminance channel W information and RGB color image information. Considering the spatial characteristics of the RGBW filter array, and based on the smoothness of the residual, a residual interpolation algorithm is designed with progressively increasing sampling rate and iterative pixel-by-pixel calculation to achieve RGBW array color reconstruction.

[0006] The objective of this invention is achieved through the following technical solution:

[0007] The RGBW array color reconstruction method based on guided filter interpolation disclosed in this invention includes the following steps:

[0008] Step 1: Reconstruct the brightness W at the position of pixel G in the initial filter array. G Pixel;

[0009] The specific implementation method of step one is as follows:

[0010] Step 1.1: Perform iterative directional interpolation at the G channel pixels in the vertical direction of the W channel using the RI method and the MLRI method respectively;

[0011] Vertical linear interpolation was performed on both the W and G channels to obtain the initial interpolation results:

[0012]

[0013] In the formula, and This represents the interpolated pixel value. The superscript v indicates the vertical direction, and the subscript (i,j) indicates the initial interpolation result at pixel (i,j).

[0014] Substitute the initial interpolation result into equation (2) to begin the iterative process; in each iteration, transform the result of the previous iteration, from... A provisional estimate was obtained. Depend on A provisional estimate was obtained. Represented as:

[0015]

[0016] In the formula, (k g ,b g ) and (k w ,b w () represents the transformation coefficient, which must be minimized; the subscript k represents the number of iterations;

[0017] Within the local window, RI and MLRI are calculated by minimizing the difference coefficient. and The minimum transform coefficients are calculated as follows:

[0018]

[0019] In this method, k is calculated using the corresponding RI formulas in equations (3) and (4). g,ri b g,ri k w,ri b w,ri This leads to equation (2). and The MLRI method calculates k using the corresponding MLRI formulas in equations (3) and (4). g,mlri b g ,mlri k w,mlri b w,mlri This leads to equation (2). and In the subsequent formula derivation, the RI method and the MLRI method are calculated separately. and The results are then combined; in subsequent formulas, both methods are represented by the superscript x (x = ri / mlri);

[0020] The residuals between the G channel and the W channel are expressed as follows:

[0021]

[0022] Interpolating the residuals, we can express it as follows:

[0023]

[0024] Add the interpolated residual to the provisional estimate to obtain the interpolation result for the k-th W channel. and Represented as:

[0025]

[0026] Step 1.2: Based on the obtained interpolation results, update and iterate the provisional estimate according to equation (8);

[0027] The number of iterations is selected based on the difference value of each pixel.

[0028]

[0029] In the formula, the function g(·) represents spatial Gaussian smoothing; the judgment criteria are as follows: The magnitude is determined by the amplitude or smoothness of the difference, when the judgment is based on... Criteria for determining whether it is greater than the previous iteration This iteration is invalid, and the iteration terminates.

[0030]

[0031] In the formula, the judgment criteria are... The magnitude of the difference Smoothness of differences composition;

[0032]

[0033] In the formula, This represents the difference between the provisional estimate of channel G at the k-th iteration and the interpolation result at the (k-1)-th iteration; This represents the difference between the provisional estimate of channel W at the k-th iteration and the interpolation result at the (k-1)-th iteration;

[0034] Step 1.3: Combine the interpolation results of the RI method and the MLRI method, and represent them as follows:

[0035]

[0036] in,

[0037]

[0038] In the formula, and These are the weight values ​​for the RI and MLRI methods, respectively;

[0039] According to equations (1) to (12), the brightness W at the pixel position of the initial filter array G is obtained. G Pixel.

[0040] Step 2: Reconstruct the brightness W at pixel B of the initial filter array.B Pixel;

[0041] The specific implementation method of step two is as follows: iterative directional interpolation is performed at the B channel pixel in the horizontal direction of the W channel using the RI method and the MLRI method respectively; the interpolation result is obtained by equation (13), the number of iterations for each pixel is determined according to equation (14), the update of equation (13) is completed, and the interpolation results of RI and MLRI are combined to obtain the brightness W at the B pixel position of the initial filter array. B Pixels

[0042]

[0043] In the formula, and This represents the interpolated pixel value. The superscript h indicates the horizontal direction, the superscript x indicates the RI or MLRI method (x = ri / mlri), and the subscript (i,j), k represents the k-th interpolation result at pixel (i,j). and These represent the residuals of channel B and channel W, respectively. and These represent the provisional estimates for channel B and channel W, respectively.

[0044]

[0045] In the formula, For a suitable number of iterations, the function g(·) represents spatial Gaussian smoothing. As a basis for judgment;

[0046]

[0047] In the formula, ri and mlri represent the results of RI and MLRI, respectively. and For weight values, This is the final interpolation result;

[0048] According to equations (13) to (15), the brightness W at the pixel position of the initial filter array B is obtained. B Pixel.

[0049] Step 3: Reconstruct the brightness W at the R pixel position of the initial filter array. R Pixel;

[0050] The specific implementation method of step three is as follows: using the RI method and the MLRI method respectively, in W G Iterative directional interpolation is performed at the R channel pixels in the horizontal direction of the pixel, at W... BIterative directional interpolation is performed at the R channel pixels in the vertical direction of the pixel; the interpolation result is obtained through equation (16), the number of iterations for each pixel is determined according to equation (17), the update of equation (16) is completed, the interpolation results of RI and MLRI are combined, and the horizontal interpolation result and the vertical interpolation result are combined to obtain the brightness W at the R pixel position of the initial filter array. R Pixels

[0051]

[0052] In the formula, and This represents the interpolated pixel value. The superscript h indicates the horizontal direction, the superscript v indicates the vertical direction, the superscript x indicates the RI or MLRI method (x = ri / mlri), and the subscript (i,j),k indicates the k-th interpolation result at pixel (i,j). and These represent the residuals of the R channel and the W channel in the horizontal and vertical directions, respectively. and These represent the provisional estimates for the R and W channels, respectively.

[0053]

[0054] In the formula, These represent the appropriate number of iterations in the horizontal and vertical directions, respectively. The function g(·) represents spatial Gaussian smoothing. These are the criteria for judgment in the horizontal and vertical directions, respectively.

[0055]

[0056] In the formula, ri and mlri represent the results of RI and MLRI, respectively. and The horizontal weight value. and The weight value is in the vertical direction. This is the final interpolation result;

[0057] According to equations (16) to (18), the brightness W at the pixel position of the initial filter array R is obtained. R Pixel.

[0058] Step 4: Reconstruct W using steps 1, 2, and 3. G W B and W R Pixel, combined with the brightness pixel W at the pixel position of the initial filter array W W The W-channel image is obtained; the W-channel image is used as the guide image for subsequent steps, and the pixels of the W-channel image are W(i,j);

[0059] Step 5: Reconstruct the G at pixel position W of the initial filter array under the guidance of the guiding image in Step 4. W R at pixel B pixel position B B at pixel, R pixel position R Pixel;

[0060] The specific implementation method of step five is as follows:

[0061] Step 5.1: Perform iterative directional interpolation at the W channel pixels in the vertical direction of the G channel using the RI method and the MLRI method respectively;

[0062] Perform vertical linear interpolation on the G channel to obtain the initial interpolation result:

[0063]

[0064] In the formula, This represents the interpolated pixel value. The superscript v indicates the vertical direction, and the subscript (i,j) indicates the initial interpolation result at pixel (i,j).

[0065] Substitute the initial interpolation result into equation (20) to begin the iterative process; in each iteration, transform the result of the previous iteration, and guide the pixel of the image by step four. A provisional estimate was obtained. Represented as:

[0066]

[0067] In the formula, (k g ,b g () represents the transformation coefficient, which must be minimized; the subscript k represents the number of iterations;

[0068] Within the local window, RI and MLRI are calculated by minimizing the difference coefficient. The minimum transform coefficients are calculated as follows:

[0069]

[0070] Among them, the RI method uses the corresponding RI formula in equation (21) to calculate k. g,ri b g,ri Thus, we obtain equation (20).

[0071] The MLRI method uses the corresponding MLRI formula in equation (21) to calculate k. g,mlri b g,mlri Thus, we obtain equation (20).

[0072] In the subsequent formula derivation, the two methods are used to calculate the corresponding values ​​respectively. and And finally combine the results;

[0073] In subsequent formulas, both methods are represented by the superscript x (x = ri / mlri);

[0074] The residual calculation for channel G is expressed as follows:

[0075]

[0076] Interpolating the residuals, we can express it as follows:

[0077]

[0078] Add the interpolated residual to the provisional estimate to obtain the interpolation result for the k-th channel G. and Represented as:

[0079]

[0080] Step 5.2: Based on the obtained interpolation results, update and iterate the provisional estimate according to equation (20);

[0081] The number of iterations is selected based on the difference value of each pixel.

[0082]

[0083] In the formula, the function g(·) represents spatial Gaussian smoothing; the judgment criteria are as follows: The magnitude is determined by the amplitude or smoothness of the difference, when the judgment is based on... Criteria for determining whether it is greater than the previous iteration This iteration is invalid, and the iteration terminates.

[0084]

[0085] In the formula, the judgment criteria are... The magnitude of the difference Smoothness of differences composition;

[0086]

[0087] In the formula, This represents the difference between the provisional estimate of channel G at the k-th iteration and the interpolation result at the (k-1)-th iteration;

[0088] Step 5.3: Combine the interpolation results of the RI method and the MLRI method, as follows:

[0089]

[0090] in,

[0091]

[0092] In the formula, and These are the weight values ​​for the RI and MLRI methods, respectively;

[0093] According to equations (19) to (29), the G at the pixel position of the initial filter array W is obtained. W Pixel;

[0094] According to formula (30), the R at the pixel position of the initial filter array B is obtained. B Pixel, according to formula (31), the initial filter array R pixel position B R Pixel.

[0095]

[0096] Step Six: Reconstruct the B pixel position of the initial filter array W under the guidance of the guiding image in Step Four. W R at pixel, G pixel position G G at pixel, R pixel position R Pixel;

[0097] The implementation method of step six is ​​as follows: performing iterative directional interpolation at the W channel pixel in the horizontal direction of the B channel using the RI method and the MLRI method respectively; obtaining the interpolation result through equation (32), determining the number of iterations for each pixel according to equation (33), completing the update of equation (32), and combining the interpolation results of RI and MLRI to obtain the B channel pixel at the W pixel position of the initial filter array. W Pixel;

[0098]

[0099] In the formula, This represents the interpolated pixel value. The superscript h indicates the horizontal direction, the superscript x indicates the RI or MLRI method (x = ri / mlri), and the subscript (i,j), k represents the k-th interpolation result at pixel (i,j). This represents the residual of channel B. This represents a provisional estimate for channel B.

[0100]

[0101] In the formula, For a suitable number of iterations, the function g(·) represents spatial Gaussian smoothing. As a basis for judgment;

[0102]

[0103] In the formula, ri and mlri represent the results of RI and MLRI, respectively. and For weight values, This is the final interpolation result;

[0104] According to equations (32) to (34), the B at the pixel position of the initial filter array W is obtained. W Pixel;

[0105] According to formula (35), the R at the position of pixel G in the initial filter array is obtained. G Pixel, according to formula (36), the initial filter array R pixel position G R Pixel.

[0106]

[0107] Step 7: Reconstruct the R at the initial filter array W pixel position under the guidance of the guiding image in Step 4. W B at pixel, G pixel position G G at pixel, B pixel position B Pixel;

[0108] Step seven is implemented by using the RI method and the MLRI method respectively, in R B Iterative directional interpolation is performed at the W channel pixels in the horizontal direction of the pixel, at R... G Iterative directional interpolation is performed at the W channel pixels in the vertical direction of the pixel; the interpolation result is obtained through equation (37), the number of iterations for each pixel is determined according to equation (38), the update of equation (37) is completed, the interpolation results of RI and MLRI are combined, and the horizontal interpolation result and the vertical interpolation result are combined to obtain the R at the W pixel position of the initial filter array. W Pixels:

[0109]

[0110] In the formula, This represents the interpolated pixel value. The superscript h indicates the horizontal direction, the superscript v indicates the vertical direction, the superscript x indicates the RI or MLRI method (x = ri / mlri), and the subscript (i,j),k indicates the k-th interpolation result at pixel (i,j). and This represents the residuals of the R channel in the horizontal and vertical directions. This represents a provisional estimate of the R channel value;

[0111]

[0112] In the formula, These represent the appropriate number of iterations in the horizontal and vertical directions, respectively. The function g(·) represents spatial Gaussian smoothing. These are the criteria for judgment in the horizontal and vertical directions, respectively.

[0113]

[0114] In the formula, ri and mlri represent the results of RI and MLRI, respectively. and The horizontal weight value. and The weight value is in the vertical direction. This is the final interpolation result;

[0115] According to equations (37) to (39), the R at the pixel position of the initial filter array W is obtained. W Pixel;

[0116] According to formula (40), the B at the position of pixel G in the initial filter array is obtained. G Pixel, according to formula (41), the initial filter array B pixel position G B Pixel.

[0117]

[0118] Step 8: Based on the R obtained from the above steps G R B and R W Reconstruct the R channel image; G R G B and G W Reconstruct the G channel image; B R B G and B W Reconstruct the B channel image; W R W G and W B The W-channel image is reconstructed, achieving RGBW array color reconstruction.

[0119] Beneficial effects:

[0120] 1. The present invention discloses an RGBW array color reconstruction method based on guided filter interpolation. Based on the color ratio law and the characteristic of guided filtering to preserve image detail areas, the grayscale correlation between the brightness channel W information and the RGB color image information is explored to realize color reconstruction of RGBW CFA mosaic images.

[0121] 2. The present invention discloses an RGBW array color reconstruction method with a gradually increasing sampling rate. Based on the spatial characteristics of the RGBW filter array and the smoothness of the residual, a residual interpolation algorithm with a gradually increasing sampling rate and pixel-by-pixel iteration is designed. This fully compensates for the defects of sparse and uneven distribution of the initial proportion of each channel, thereby increasing the accuracy of image reconstruction of each channel.

[0122] 3. The present invention discloses a pixel-by-pixel iterative feedback RGBW array color reconstruction method. Unlike the traditional method where the iteration termination condition is a global threshold, the method of the present invention can obtain the optimal iteration termination condition for each pixel according to the pixel-by-pixel evaluation factor, so that each pixel achieves optimal iteration, and the final reconstructed image is more in line with the requirements of human eye observation, realizing natural color imaging. Attached Figure Description

[0123] Figure 1 Diagrams showing the Bayer color filter array layout and the RGBW color filter array layout.

[0124] Figure 2 This is a flowchart of an RGBW array color reconstruction method based on guided filter interpolation according to the present invention.

[0125] Figure 3 This is a flowchart of a bidirectional guided filtering interpolation process based on RGBW CFA according to the present invention.

[0126] Figure 4 This is a flowchart of a unidirectional guided filtering interpolation process based on RGBW CFA according to the present invention.

[0127] Figure 5 This is the Kodak dataset, which serves as the original reference image for method verification in this invention.

[0128] Figure 6 This is the mosaic image dataset corresponding to the Kodak dataset, which is the reference image dataset of this invention.

[0129] Figure 7 This is a diagram showing the effect of color reconstruction of a mosaic image dataset using the method of the present invention. Detailed Implementation

[0130] To better illustrate the purpose and advantages of the present invention, the invention will be further described below in conjunction with the accompanying drawings and examples.

[0131] Example:

[0132] To verify the feasibility of this method, we used it to perform color reconstruction on the mosaicked Kodak dataset images, and then verified the method from both a subjective human perspective and an objective evaluation metric.

[0133] The reference images are from the Kodak dataset, which is widely used in image processing, such as... Figure 5 As shown. The Kodak dataset images after adding mosaic are as follows. Figure 6 As shown.

[0134] like Figure 1 As shown in the figure, this embodiment discloses an RGBW array color reconstruction method based on guided filter interpolation. The specific implementation steps are as follows:

[0135] Step 1: Reconstruct the brightness W at the position of pixel G in the initial filter array. G Pixel;

[0136] The specific implementation method of step one is as follows:

[0137] Step 1.1: Perform iterative directional interpolation at the G channel pixels in the vertical direction of the W channel using the RI method and the MLRI method respectively;

[0138] First, vertical linear interpolation is performed on both the W and G channels to obtain the initial interpolation results:

[0139]

[0140] In the formula, and This represents the interpolated pixel value. The superscript v indicates the vertical direction, and the subscript (i,j) indicates the initial interpolation result at pixel (i,j).

[0141] Substitute the initial interpolation result into equation (2) to begin the iterative process; in each iteration, transform the result of the previous iteration, from... A provisional estimate was obtained. Depend on A provisional estimate was obtained. Represented as:

[0142]

[0143] In the formula, (k g ,b g ) and (k w ,b w () represents the transformation coefficient, which must be minimized; the subscript k represents the number of iterations;

[0144] Within the local window, RI and MLRI are calculated by minimizing the difference coefficient. and The minimum transform coefficients are calculated as follows:

[0145]

[0146] In this method, k is calculated using the corresponding RI formulas in equations (3) and (4). g,ri b g,ri k w,ri b w,ri This leads to equation (2). and The MLRI method calculates k using the corresponding MLRI formulas in equations (3) and (4). g,mlri b g ,mlri k w,mlri b w,mlri This leads to equation (2). and In the subsequent formula derivation, the two methods are used to calculate the corresponding values ​​respectively. and The results are then combined at the end; in subsequent formulas, both methods are represented by the superscript x (x = ri / mlri);

[0147] The residuals between the G channel and the W channel are expressed as follows:

[0148]

[0149] Interpolating the residuals, we can express it as follows:

[0150]

[0151] Finally, the interpolated residual is added to the provisional estimate to obtain the interpolation result for the k-th W channel. and Represented as:

[0152]

[0153] Step 1.2: Based on the obtained interpolation results, update and iterate the provisional estimate according to equation (8);

[0154] The number of iterations is selected based on the difference value of each pixel.

[0155]

[0156] In the formula, the function g(·) represents spatial Gaussian smoothing; the judgment criteria are as follows: The magnitude is determined by the amplitude or smoothness of the difference, when the judgment is based on... Criteria for determining whether it is greater than the previous iteration This iteration is invalid, and the iteration terminates.

[0157]

[0158] In the formula, the judgment criteria are... The magnitude of the difference Smoothness of differences composition;

[0159]

[0160] In the formula, This represents the difference between the provisional estimate of channel G at the k-th iteration and the interpolation result at the (k-1)-th iteration; This represents the difference between the provisional estimate of channel W at the k-th iteration and the interpolation result at the (k-1)-th iteration;

[0161] Step 1.3: Combine the interpolation results of RI and MLRI, and represent them as follows:

[0162]

[0163] in,

[0164]

[0165] In the formula, and These are the weight values ​​for the RI and MLRI methods, respectively;

[0166] According to equations (1) to (12), the brightness W at the pixel position of the initial filter array G is obtained. G Pixel.

[0167] Step 2: Reconstruct the brightness W at pixel B of the initial filter array. B Pixel;

[0168] The specific implementation method of step two is as follows: iterative directional interpolation is performed at the B channel pixel in the horizontal direction of the W channel using the RI method and the MLRI method respectively; the interpolation result is obtained by equation (13), the number of iterations for each pixel is determined according to equation (14), the update of equation (13) is completed, and the interpolation results of RI and MLRI are combined to obtain the brightness W at the B pixel position of the initial filter array. B Pixels

[0169]

[0170] In the formula, and This represents the interpolated pixel value. The superscript h indicates the horizontal direction, the superscript x indicates the RI or MLRI method (x = ri / mlri), and the subscript (i,j), k represents the k-th interpolation result at pixel (i,j). and These represent the residuals of channel B and channel W, respectively. and These represent the provisional estimates for channel B and channel W, respectively.

[0171]

[0172] In the formula, For a suitable number of iterations, the function g(·) represents spatial Gaussian smoothing. As a basis for judgment;

[0173]

[0174] In the formula, ri and mlri represent the results of RI and MLRI, respectively. and For weight values, This is the final interpolation result;

[0175] According to equations (13) to (15), the brightness W at the pixel position of the initial filter array B is obtained. B Pixel.

[0176] Step 3: Reconstruct the brightness W at the R pixel position of the initial filter array. R Pixel;

[0177] The specific implementation method of step three is as follows: using the RI method and the MLRI method respectively, in W G Iterative directional interpolation is performed at the R channel pixels in the horizontal direction of the pixel, at W... B Iterative directional interpolation is performed at the R channel pixels in the vertical direction of the pixel; the interpolation result is obtained through equation (16), the number of iterations for each pixel is determined according to equation (17), the update of equation (16) is completed, the interpolation results of RI and MLRI are combined, and the horizontal interpolation result and the vertical interpolation result are combined to obtain the brightness W at the R pixel position of the initial filter array. R Pixels

[0178]

[0179] In the formula, and This represents the interpolated pixel value. The superscript h indicates the horizontal direction, the superscript v indicates the vertical direction, the superscript x indicates the RI or MLRI method (x = ri / mlri), and the subscript (i,j),k indicates the k-th interpolation result at pixel (i,j). and These represent the residuals of the R channel and the W channel in the horizontal and vertical directions, respectively. and These represent the provisional estimates for the R and W channels, respectively.

[0180]

[0181] In the formula, These represent the appropriate number of iterations in the horizontal and vertical directions, respectively. The function g(·) represents spatial Gaussian smoothing. These are the criteria for judgment in the horizontal and vertical directions, respectively.

[0182]

[0183] In the formula, ri and mlri represent the results of RI and MLRI, respectively. and The horizontal weight value. and The weight value is in the vertical direction. This is the final interpolation result;

[0184] According to equations (16) to (18), the brightness W at the pixel position of the initial filter array R is obtained. R Pixel.

[0185] Step 4: The W reconstructed from steps 1, 2, and 3 G W B and W R Pixel, combined with the brightness pixel W at the pixel position of the initial filter array W W The W-channel image is obtained; the W-channel image is used as the guide image for subsequent steps, and the pixels of the W-channel image are W(i,j);

[0186] Step 5: Reconstruct the G at pixel position W of the initial filter array under the guidance of the guiding image in Step 4. W R at pixel B pixel position B B at pixel, R pixel position R Pixel;

[0187] The specific implementation method of step five is as follows:

[0188] Step 5.1: Perform iterative directional interpolation at the W channel pixels in the vertical direction of the G channel using the RI method and the MLRI method respectively;

[0189] First, vertical linear interpolation is performed on the G channel to obtain the initial interpolation result:

[0190]

[0191] In the formula, This represents the interpolated pixel value. The superscript v indicates the vertical direction, and the subscript (i,j) indicates the initial interpolation result at pixel (i,j).

[0192] Substitute the initial interpolation result into equation (20) to begin the iterative process; in each iteration, transform the result of the previous iteration, and guide the pixel of the image by step four. A provisional estimate was obtained. Represented as:

[0193]

[0194] In the formula, (k g ,b g () represents the transformation coefficient, which must be minimized; the subscript k represents the number of iterations;

[0195] Within the local window, RI and MLRI are calculated by minimizing the difference coefficient. The minimum transform coefficients are calculated as follows:

[0196]

[0197] Among them, the RI method uses the corresponding RI formula in equation (21) to calculate k. g,ri b g,ri Thus, we obtain equation (20).

[0198] The MLRI method uses the corresponding MLRI formula in equation (21) to calculate k. g,mlri b g,mlri Thus, we obtain equation (20).

[0199] In the subsequent formula derivation, the two methods are used to calculate the corresponding values ​​respectively. and And finally combine the results;

[0200] In subsequent formulas, both methods are represented by the superscript x (x = ri / mlri);

[0201] The residual calculation for channel G is expressed as follows:

[0202]

[0203] Interpolating the residuals, we can express it as follows:

[0204]

[0205] Finally, the interpolated residual is added to the provisional estimate to obtain the interpolation result for the k-th channel G. and Represented as:

[0206]

[0207] Step 5.2: Based on the obtained interpolation results, update and iterate the provisional estimate according to equation (20);

[0208] The number of iterations is selected based on the difference value of each pixel.

[0209]

[0210] In the formula, the function g(·) represents spatial Gaussian smoothing; the judgment criteria are as follows: The magnitude is determined by the amplitude or smoothness of the difference, when the judgment is based on... Criteria for determining whether it is greater than the previous iteration This iteration is invalid, and the iteration terminates.

[0211]

[0212] In the formula, the judgment criteria are... The magnitude of the difference Smoothness of differences composition;

[0213]

[0214] In the formula, This represents the difference between the provisional estimate of channel G at the k-th iteration and the interpolation result at the (k-1)-th iteration;

[0215] Step 5.3: Combine the interpolation results of RI and MLRI, as follows:

[0216]

[0217] in,

[0218]

[0219] In the formula, and These are the weight values ​​for the RI and MLRI methods, respectively;

[0220] According to equations (19) to (29), the G at the pixel position of the initial filter array W is obtained. W Pixel;

[0221] Similarly, according to formula (30), the R at the pixel position of the initial filter array B is obtained. B The pixel B at the initial filter array R pixel position is obtained according to formula (31). R Pixel.

[0222]

[0223] Step Six: Reconstruct the B pixel position of the initial filter array W under the guidance of the guiding image in Step Four. W R at pixel, G pixel position G G at pixel, R pixel position R Pixel;

[0224] Step six is ​​implemented by performing iterative directional interpolation at the W channel pixel in the horizontal direction of the B channel using both the RI and MLRI methods. The interpolation result is obtained using equation (30), and the number of iterations for each pixel is determined according to equation (31) to update equation (30). The interpolation results of RI and MLRI are then combined to obtain the B channel pixel at the W pixel position of the initial filter array. W Pixel;

[0225]

[0226] In the formula, This represents the interpolated pixel value. The superscript h indicates the horizontal direction, the superscript x indicates the RI or MLRI method (x = ri / mlri), and the subscript (i,j), k represents the k-th interpolation result at pixel (i,j). This represents the residual of channel B. This represents a provisional estimate for channel B.

[0227]

[0228] In the formula, For a suitable number of iterations, the function g(·) represents spatial Gaussian smoothing. As a basis for judgment;

[0229]

[0230] In the formula, ri and mlri represent the results of RI and MLRI, respectively. and For weight values, This is the final interpolation result;

[0231] According to equations (32) to (34), the B at the pixel position of the initial filter array W is obtained. W Pixel;

[0232] Similarly, according to formula (35), the R at the pixel position of the initial filter array G is obtained. G The pixel is obtained according to formula (36) at the pixel position of the initial filter array R. R Pixel.

[0233]

[0234] Step 7: Reconstruct the R at the initial filter array W pixel position under the guidance of the guiding image in Step 4. W B at pixel, G pixel position G G at pixel, B pixel position B Pixel;

[0235] Step seven is implemented by using the RI method and the MLRI method respectively, in R B Iterative directional interpolation is performed at the W channel pixels in the horizontal direction of the pixel, at R... G Iterative directional interpolation is performed at the W channel pixels in the vertical direction of the pixel; the interpolation result is obtained through equation (33), the number of iterations for each pixel is determined according to equation (34), the update of equation (33) is completed, the interpolation results of RI and MLRI are combined, and the horizontal interpolation result and the vertical interpolation result are combined to obtain the R at the W pixel position of the initial filter array. W Pixels:

[0236]

[0237] In the formula, This represents the interpolated pixel value. The superscript h indicates the horizontal direction, the superscript v indicates the vertical direction, the superscript x indicates the RI or MLRI method (x = ri / mlri), and the subscript (i,j),k indicates the k-th interpolation result at pixel (i,j). and This represents the residuals of the R channel in the horizontal and vertical directions. This represents a provisional estimate of the R channel value;

[0238]

[0239] In the formula, These represent the appropriate number of iterations in the horizontal and vertical directions, respectively. The function g(·) represents spatial Gaussian smoothing. These are the criteria for judgment in the horizontal and vertical directions, respectively.

[0240]

[0241] In the formula, ri and mlri represent the results of RI and MLRI, respectively. and The horizontal weight value. and The weight value is in the vertical direction. This is the final interpolation result;

[0242] According to equations (37) to (39), the R at the pixel position of the initial filter array W is obtained. W Pixel;

[0243] Similarly, according to formula (40), the B at the pixel position of the initial filter array G is obtained. G The pixel is obtained according to formula (41) at the pixel position of the initial filter array B. B Pixel.

[0244]

[0245] Step 8: Based on the R obtained from the above steps G R B and R W Reconstruct the R channel image; G R G B and G W Reconstruct the G channel image; B R B G and B W Reconstruct the B channel image; W R W G and W B The W-channel image is reconstructed, achieving RGBW array color reconstruction. The final reconstructed color image is as follows: Figure 7 As shown.

[0246] To evaluate the sharpness and color accuracy of this method on test images, a quantitative analysis was performed on aspects such as color performance and sharpness. When a reference image is available, Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), Color Angle Error (AE), and CIE 1976L were used. * a * b * Color space color difference (ΔE) ab To verify the color reconstruction effect, considering that a reference image is often absent in actual imaging, normalized variance (NV), Tenenbaum gradient, and Brenner gradient are introduced as parameters to measure image sharpness. Bilinear interpolation is a classic demosaic algorithm, and alternating projection (AP) is a further optimization of the Bilinear method. The SONY-RGBW array reconstruction method based on residuals and high-frequency replacement (RIHR-CRM) has been a method with superior residual interpolation performance in the past three years. Therefore, Bilinear, AP, and RIHR-CRM methods are used for color reconstruction. The method in this embodiment is compared and evaluated with the Bilinear, AP, and RIHR-CRM methods. Table 1 shows the average objective evaluation results on the Kodak dataset. It can be seen that the method in this embodiment performs well in PSNR, SSIM, AE, and ΔE. ab The method outperforms other methods in terms of NV, Tenenbaum gradient, and Brenner gradient metrics. The bolded values ​​in Table 1 represent the optimal values ​​for these metrics.

[0247] Table 1. Average Objective Results of the Kodak Dataset

[0248]

[0249] This method addresses the issue of RGBW CFA images where each pixel possesses only single-channel information, necessitating color reconstruction. It proposes an RGBW array color reconstruction method based on guided filter interpolation, capable of reconstructing the color of RGBW CFA mosaic images, restoring clear details, and effectively reducing color aliasing and jagged edges. In terms of objective evaluation metrics for sharpness and color reproduction, this method demonstrates superior performance compared to several classic algorithms.

[0250] The above detailed description further illustrates the purpose, technical solution, and beneficial effects of the invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method of RGBW array color reconstruction based on guided filter interpolation, characterized in that: The method comprises the following steps, Step one, reconstruct the luminance W at the pixel position of the initial filter array G G pixel; The specific implementation method of step one is: Step 1.1, respectively through the RI method and the MLRI method, iterative direction interpolation is performed at the G channel pixels in the vertical direction of the W channel; The vertical linear interpolation is performed on the W channel and the G channel to obtain an initial interpolation result: wherein and denotes the interpolated pixel value, the superscript v denotes the vertical direction, the subscript (i,j)0 denotes the initial interpolation result at the pixel point (i,j). The initial interpolation result is brought into equation (2) to start the iteration process; in each iteration, the last iteration result is transformed to get a tentative estimate from a tentative estimate is expressed as: where (k g ,b g ) and (k w ,b w ) represent the transform coefficients, and the minimum is required to be guaranteed; the subscript k represents the iteration number; In the local window, the RI and the MLRI are computed by minimizing the difference coefficient and to the minimum transform coefficient, denoted as: wherein the RI method uses the corresponding RI formula in equation (3) and equation (4) to calculate k g,ri , b g,ri , k w,ri , b w,ri and further obtains the k and The MLRI method uses the corresponding MLRI formula in equation (3) and equation (4) to calculate k g,mlri , b g,mlri , k w,mlri , b w,mlri and further obtains the k and In the subsequent formula derivation, the RI method and the MLRI method each calculate the corresponding and and combine the results; in the subsequent formula, the two methods are denoted as superscript x, x = ri / mlri; The residual calculation of the G channel and the W channel is expressed as: The interpolation of the residual is expressed as: adding the interpolated residual to the tentative estimate to obtain an interpolated result for the kth W channel and is represented as: Step 1.2, according to the obtained interpolation result, the provisional estimation is updated and iterated according to formula (8); selecting the number of iterations according to the difference value of each pixel itself In the formula, the function g(·) represents spatial Gaussian smoothing; the judgment basis is determined by the amplitude or smoothness of the difference, and when the current judgment basis is greater than the judgment basis of the last iteration, the current iteration is invalid, and the iteration is terminated. In the formula, the basis for judgment consists of the magnitude of the difference and the smoothness of the difference ; wherein denotes the difference between the kth tentative estimate of the G channel and the interpolation result of the k-1th iteration; denotes the difference between the kth tentative estimate of the W channel and the interpolation result of the k-1th iteration; Step 1.3, the interpolation results of the RI method and the MLRI method are combined, expressed as Wherein, wherein and are the weight values for the RI and MLRI methods, respectively. According to the formula (1) to formula (12), the luminance W at the pixel position of the initial filter array G is obtained G pixel; Step two, reconstructing the luminance W at the pixel position of the initial filter array B B pixel; Step three, reconstructing the luminance W at the R pixel position of the initial filter array R pixel; Step four, W channel image is obtained by reconstructing W G , W B and W R pixel at the position of the initial filter array W pixel, combined with the brightness pixel W W , and the pixel of the W channel image is W(i,j). Step five, reconstruct G at the W pixel location of the initial filter array under the guidance of the guided image from step four W pixel, R at the B pixel location B pixel, B at the R pixel location R pixel; Step six, reconstruct B at the initial filter array W pixel location under the guidance of the guided image from step four W R at the R pixel location, G at the G pixel location, and B at the B pixel location G G at the R pixel location, R at the R pixel location, and B at the B pixel location R pixel; Step seven, reconstruct R at the initial filter array W pixel location under the guidance of the guided image from step four W B at the G pixel location G G at the B pixel location B pixel; Step eight, R G , R B , and R W reconstruct the R channel image; G R , G B , and G W reconstruct the G channel image; B R , B G , and B W reconstruct the B channel image; and W R , W G , and W B reconstruct the W channel image, realizing RGBW array color reconstruction. 2.The RGBW array color reconstruction method based on guided filter interpolation of claim 1, wherein: The specific implementation method of step two is that, respectively through the RI method and the MLRI method, iterative direction interpolation is performed at the B channel pixels in the horizontal direction of the W channel; the interpolation result is obtained through formula (13), the iteration number of each pixel is determined according to formula (14), the update of formula (13) is completed, the interpolation results of the RI and the MLRI are combined, and the luminance W at the initial filter array B pixel position is obtained B pixel In the formula, and denote the interpolated pixel values, the superscript h denotes the horizontal direction, the superscript x denotes the RI or MLRI method x, x = ri / mlri, and the subscript (i, j), k denotes the kth interpolation result at the pixel point (i, j), and respectively denote the residual errors of the B channel and the W channel, and respectively denote the provisional estimated values of the B channel and the W channel. wherein is a suitable number of iterations, and g(·) is a spatial Gaussian smoothing function, is a decision criterion; wherein, ri and mlri represent the results of RI and MLRI, respectively, and is a weight value, is the final interpolation result; According to equations (13) to (15), the luminance W at the pixel position of the initial filter array B is obtained B pixel.

3. The RGBW array color reconstruction method based on guided filter interpolation of claim 2, wherein: The specific implementation method of step three is: respectively through the RI method and the MLRI method, in W G Iterative directional interpolation is performed at R channel pixels in the horizontal direction of the pixel, and the interpolation result is obtained through formula (16) B Iterative directional interpolation is performed at R channel pixels in the vertical direction of the pixel; the interpolation result is obtained through formula (16), the iteration number of each pixel is determined according to formula (17), the update of formula (16) is completed, the interpolation results of RI and MLRI are combined, and the horizontal directional interpolation result and the vertical directional interpolation result are integrated to obtain the luminance at the R pixel position of the initial filter array R pixel wherein, and denote the interpolated pixel values, the superscript h denotes the horizontal direction, the superscript v denotes the vertical direction, the superscript x denotes the RI or MLRI method x, x = ri / mlri, and the subscript (i,j), k denotes the kth interpolation result at the pixel point (i,j), and denote the horizontal and vertical residuals of the R and W channels, respectively, and denote the provisional estimates of the R and W channels, respectively. wherein are the appropriate number of iterations in the horizontal and vertical directions, respectively, and g(·) denotes a spatial Gaussian smoothing, are the decision criteria in the horizontal and vertical directions, respectively; wherein, ri and mlri represent the results of RI and MLRI, respectively, and is a horizontal direction weight value, and is a vertical direction weight value, is a final interpolation result; According to equations (16) to (18), the luminance W at the pixel position of the initial filter array R is obtained R pixel.

4. The RGBW array color reconstruction method based on guided filter interpolation of claim 3, wherein: The specific implementation method of step five is: Step 5.1, respectively through the RI method and the MLRI method, iterative direction interpolation is performed at the G channel pixels in the vertical direction of the W channel; The vertical linear interpolation is performed on the G channel to obtain an initial interpolation result: In the formula, represents the interpolated pixel value, the superscript v represents the vertical direction, the subscript (i, j)0 represents the initial interpolation result at the pixel point (i, j). The initial interpolation result is brought into equation (20) to start the iteration process; in each iteration, the pixel of the guided image is transformed from the last iteration result by step four A tentative estimate is obtained is represented as: wherein (k g ,b g ) denotes the transform coefficients, and the minimum is to be ensured; the subscript k denotes the iteration number; In the local window, the RI and the MLRI are computed by minimizing the difference coefficient to the minimum transform coefficient, denoted as: where k is calculated using the corresponding RI formula in equation (21) g,ri , b g,ri , and further k in equation (20) MLRI method calculates k using the corresponding MLRI formula in equation (21) g,mlri , b g,mlri , and further k in equation (20) In the following formula derivation, each method calculates the corresponding and and combines the results at the end; in the following formulas, the two methods are denoted by the superscript x, x = ri / mlri; The residual calculation of the G channel is expressed as: The interpolation of the residual is expressed as: adding the interpolated residual to the tentative estimate to obtain an interpolated result for the kth G channel and is represented as: Step 5.2, according to the obtained interpolation result, the provisional estimation is updated and iterated according to formula (20); selecting the number of iterations according to the difference value of each pixel itself In the formula, the function g(·) represents spatial Gaussian smoothing; the judgment basis is determined by the amplitude or smoothness of the difference, and when the current judgment basis is greater than the judgment basis of the last iteration , the current iteration is invalid, and the iteration is terminated. In the formula, the basis for judgment consists of the magnitude of the difference and the smoothness of the difference ; wherein represents the difference between the kth tentative estimate of the G channel and the interpolation result of the k-1th Step 5.3, the interpolation results of the RI method and the MLRI method are combined, expressed as: Wherein, wherein and are the weight values for the RI and MLRI methods, respectively. According to equations (19) to (29), G at the pixel position of the initial filter array W is obtained W pixel; R at the pixel position of the initial filter array B according to equation (30) B B at the pixel position of the initial filter array R according to equation (31) R pixel; 5. The RGBW array color reconstruction method based on guided filter interpolation of claim 4, wherein: The implementation method of step six is: respectively through the RI method and the MLRI method, iterative direction interpolation is performed at the W channel pixels in the horizontal direction of the B channel; the interpolation result is obtained through formula (32), the iteration number of each pixel is determined according to formula (33), the update of formula (32) is completed, the interpolation results of the RI and the MLRI are combined, and the B W pixels at the initial filter array W pixel positions are obtained In the formula, represents the interpolated pixel value, the superscript h represents the horizontal direction, the superscript x represents the RI or MLRI method x, x = ri / mlri, and the subscript (i,j), k represents the kth interpolation result at the pixel point (i,j), represents the residual of the B channel, represents the provisional estimated value of the B channel; wherein is a suitable number of iterations, and g(·) is a spatial Gaussian smoothing function, is a decision criterion; wherein, ri and mlri represent the results of RI and MLRI, respectively, and is a weight value, is the final interpolation result; According to equations (32) to (34), B at the W pixel position of the initial filter array is obtained W pixel; R at the pixel position of the initial filter array G according to equation (35) G G at the pixel position of the initial filter array R according to equation (36) R pixel; 6.The RGBW array color reconstruction method based on guided filter interpolation of claim 5, wherein: Step seven is implemented by using the RI method and the MLRI method respectively, in R B Iterative directional interpolation is performed at the W channel pixels in the horizontal direction of the pixel, at R... G Iterative directional interpolation is performed at the W channel pixels in the vertical direction of the pixel; the interpolation result is obtained through equation (37), the number of iterations for each pixel is determined according to equation (38), the update of equation (37) is completed, the interpolation results of the RI method and the MLRI method are combined, and the horizontal interpolation result and the vertical interpolation result are combined to obtain the R at the W pixel position of the initial filter array. W Pixels: wherein, denotes the interpolated pixel value, the superscript h denotes the horizontal direction, the superscript v denotes the vertical direction, the superscript x denotes the RI or MLRI method x, x = ri / mlri, and the subscript (i,j), k denotes the kth interpolation result at the pixel point (i,j), and denotes the residual of the R channel in the horizontal direction and the vertical direction, denotes the provisional estimate of the R channel; wherein are the appropriate number of iterations in the horizontal and vertical directions, respectively, and g(·) denotes a spatial Gaussian smoothing, are the decision criteria in the horizontal and vertical directions, respectively; wherein, ri and mlri represent the results of RI and MLRI, respectively, and is a horizontal direction weight value, and is a vertical direction weight value, is a final interpolation result; According to equations (37) to (39), R at the W pixel position of the initial filter array is obtained W pixel; B at the pixel position of the initial filter array G according to equation (40) G G at the pixel position of the initial filter array B according to equation (41) B pixel;

Images