Method for Image Demosaicing

Abstract

The present invention provides an image demosaicing device and method, including: inputting a to-be-processed image in Bayer format; mapping the data of the red domain, green domain, and blue domain to the Log domain respectively and performing gain processing; initializing the green domain data after the gain processing; initializing the red domain data according to the initial value of the green domain; calculating a first similarity between the initial value of the red domain and the initial value of the green domain, and calculating a first residual between the initial value of the red domain and the initial value of the green domain; initializing the blue domain data according to the initial value of the green domain; calculating a second similarity between the initial value of the blue domain and the initial value of the green domain, and calculating a second residual between the initial value of the blue domain and the initial value of the green domain; calculating the final green domain data according to the initial value of the green domain, the first similarity, and the second similarity; calculating the final red domain data and the final blue domain data through the final green domain data, the first residual, and the second residual; and transforming into an RGB image.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a method for image demosaicing. Background Art

[0002] Digital color cameras are widely used in our daily lives. A color filter array (CFA) is used in the camera to collect the color information of an image. At each pixel position, only one of the three color elements of red (R) / green (G) / blue (B) is sampled. This original monochromatic image of the target object is called a Bayer image. The process of calculating the original three-channel color values of each pixel point from this format of image is called demosaicing. As people's requirements for the quality of interpolated images are gradually increasing, a more perfect demosaicing method is very important.

[0003] The existing demosaicing methods are mainly linear interpolation based on the constancy of small-area color differences and directional interpolation based on edge indication. These two types of methods have low computational complexity and good effects in homogeneous regions, but they are prone to polarization artifacts and jagged phenomena in complex texture regions, high-contrast regions, and non-directional edge regions of the image, affecting the quality of the image. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for image demosaicing, which can avoid polarization artifacts and jagged phenomena in complex texture regions, high-contrast regions, and non-directional edge regions of the image, thereby improving the quality of the image.

[0005] To achieve the above purpose, the present invention provides an image demosaicing device, including:

[0006] An image input unit, configured to input a to-be-processed image in Bayer format, where the to-be-processed image includes red domain data, green domain data, and blue domain data;

[0007] An image Log domain transformation unit, configured to map the red domain data, green domain data, and blue domain data to the Log domain respectively, and perform gain processing on the red domain data, green domain data, and blue domain data mapped to the Log domain;

[0008] A green domain initialization unit, configured to initialize the green domain data after gain processing to obtain an initial value of the green domain;

[0009] A red domain initialization unit, configured to initialize the red domain data according to the initial value of the green domain to obtain an initial value of the red domain;

[0010] A red-green similarity calculation unit, which is used to calculate the similarity between the initial value of the red domain and the initial value of the green domain to obtain a first similarity. At the same time, it is also used to calculate the residual between the initial value of the red domain and the initial value of the green domain to obtain a first residual;

[0011] A blue domain initialization unit, which is used to initialize the blue domain data according to the initial value of the green domain to obtain the initial value of the blue domain;

[0012] A blue-green similarity calculation unit, which is used to calculate the similarity between the initial value of the blue domain and the initial value of the green domain to obtain a second similarity. At the same time, it is also used to calculate the residual between the initial value of the blue domain and the initial value of the green domain to obtain a second residual;

[0013] A green domain weighted calculation unit, which is used to calculate the final green domain data according to the initial value of the green domain, the first similarity and the second similarity;

[0014] A red-blue domain weighted calculation unit, which is used to calculate the final red domain data and the final blue domain data through the final green domain data, the first residual and the second residual;

[0015] An RGB image Log domain inverse transformation unit, which is used to transform the final green domain data, the final blue domain data and the final red domain data from the Log domain into an RGB image.

[0016] Optionally, in the image demosaicing device, it further includes: an RGB image output unit, which is used to output an RGB image.

[0017] Optionally, in the image demosaicing device, the red domain data includes R channel data, the green domain data includes Gr channel data and Gb channel data, and the blue domain data includes B channel data.

[0018] Optionally, in the image demosaicing device, the image Log domain transformation unit includes:

[0019] A Log domain mapping module, which is used to map the R channel data, Gr channel data, Gb channel data and B channel data to the Log domain respectively to obtain the R channel data, Gr channel data, Gb channel data and B channel data mapped to the Log domain;

[0020] A channel data gain compensation module, which is used to perform gain on the R channel data, Gr channel data, Gb channel data and B channel data mapped to the Log domain.

[0021] Optionally, in the image demosaicing device, the green domain initialization unit includes:

[0022] The green domain 0° direction initialization module is used to initialize the processed green domain data in the 0° direction to obtain the initial value of the green domain in the 0° direction;

[0023] The green domain 90° direction initialization module is used to initialize the processed green domain data in the 90° direction to obtain the initial value of the green domain in the 90° direction.

[0024] Optionally, in the image demosaicing device, the red domain initialization unit includes:

[0025] The first 0° direction value extraction module is used to extract the initial value of the same row as the R channel in the 0° direction of the green domain and the value of the R channel itself;

[0026] The first two-dimensional wavelet filtering module is used to perform two-dimensional wavelet filtering on the initial value of the same row as the R channel in the 0° direction of the green domain to obtain the green feature component, green H component, green V component and green D component in the 0° direction, and to perform two-dimensional wavelet filtering on the value of the R channel itself to obtain the red feature component in the 0° direction;

[0027] The first red domain reference-guided filtering module is used to perform guided filtering on the green feature component to obtain the entire distribution of the red domain feature component in the 0° direction;

[0028] The first two-dimensional wavelet synthesis module is used to perform inverse wavelet operations on the green H component, green V component and green D component in the 0° direction to obtain the red initial value in the 0° direction;

[0029] The first 90° direction value extraction module is used to extract the initial value of the same column as the R channel in the 90° direction of the green domain and the value of the R channel itself;

[0030] The second two-dimensional wavelet filtering module is used to perform two-dimensional wavelet filtering on the initial value of the same column as the R channel in the 90° direction of the green domain to obtain the green feature component, green H component, green V component and green D component in the 90° direction, and to perform two-dimensional wavelet filtering on the value of the R channel itself to obtain the red feature component in the 90° direction;

[0031] The second red domain reference-guided filtering module is used to perform guided filtering on the green feature component in the 90° direction to obtain the entire distribution of the red domain feature component in the 90° direction;

[0032] The second two-dimensional wavelet synthesis module is used to perform inverse wavelet operations on the green H component, green V component and green D component in the 90° direction to obtain the red initial value in the 90° direction.

[0033] Optionally, in the image demosaicing device, the red-green similarity calculation unit includes:

[0034] A first 0° direction similarity calculation module for calculating the similarity between the red domain and the green domain in the 0° direction;

[0035] A first 90° direction similarity calculation module for calculating the similarity between the red domain and the green domain in the 90° direction.

[0036] Optionally, in the image demosaicing device, the blue domain initialization unit includes:

[0037] A second 0° direction value extraction module for extracting the initial value of the same row as the B channel in the 0° direction of the green domain and the value of the B channel itself;

[0038] A third two-dimensional wavelet filtering module for performing two-dimensional wavelet filtering on the initial value of the same row as the R channel in the 0° direction of the green domain to obtain the green feature component, green H component, green V component, and green D component in the 0° direction, and for performing two-dimensional wavelet filtering on the value of the B channel itself to obtain the blue feature component in the 0° direction;

[0039] A first blue domain reference-guided filtering module for performing guided filtering on the blue feature component in the 0° direction to obtain the entire distribution of the blue domain feature component in the 0° direction;

[0040] A third two-dimensional wavelet synthesis module for performing inverse wavelet operation on the green H component, green V component, and green D component in the 0° direction to obtain the blue initial value in the 0° direction;

[0041] A second 90° direction value extraction module for extracting the initial value of the same column as the B channel in the 90° direction of the green domain and the value of the B channel itself;

[0042] A fourth two-dimensional wavelet filtering module for performing two-dimensional wavelet filtering on the initial value of the same column as the B channel in the 90° direction of the green domain to obtain the green feature component, green H component, green V component, and green D component in the 90° direction, and for performing two-dimensional wavelet filtering on the value of the B channel itself to obtain the blue feature component in the 90° direction;

[0043] A second blue domain reference-guided filtering module for performing guided filtering on the blue feature component in the 90° direction to obtain the entire distribution of the blue domain feature component in the 90° direction;

[0044] The fourth two-dimensional wavelet synthesis module is used to perform guided filtering on the blue feature component in the 90° direction to obtain the entire distribution of the blue domain feature component in the 90° direction.

[0045] Optionally, in the image demosaicing device, the blue-green similarity calculation unit includes:

[0046] The second 0° direction similarity calculation module is used to calculate the similarity between the blue domain and the green domain in the 0° direction;

[0047] The second 90° direction similarity calculation module is used to calculate the similarity between the blue domain and the green domain in the 90° direction.

[0048] Optionally, in the image demosaicing device, the green domain weighted calculation unit includes:

[0049] The weight calculation module is used to calculate the weights of the green domain in the 00°, 90°, 180°, and 270° directions respectively according to the green domain initial value, the similarity between the blue domain and the green domain, and the similarity between the red domain and the green domain;

[0050] The weighted calculation module is used to calculate the final green domain data according to the weights of the green domain in the 00°, 90°, 180°, and 270° directions respectively.

[0051] The present invention also provides an image demosaicing method using the image demosaicing device, including:

[0052] Input a Bayer format image to be processed, where the image to be processed includes red domain data, green domain data, and blue domain data;

[0053] Map the red domain data, green domain data, and blue domain data to the Log domain respectively, and perform gain processing on the red domain data, green domain data, and blue domain data mapped to the Log domain;

[0054] Initialize the green domain data after gain processing to obtain the green domain initial value;

[0055] Initialize the red domain data according to the green domain initial value to obtain the red domain initial value;

[0056] Calculate the similarity between the red domain initial value and the green domain initial value to obtain the first similarity, and at the same time, calculate the residual between the red domain initial value and the green domain initial value to obtain the first residual;

[0057] Initialize the blue domain data according to the green domain initial value to obtain the blue domain initial value;

[0058] Calculate the similarity between the initial value of the blue domain and the initial value of the green domain to obtain a second similarity. At the same time, calculate the residual between the initial value of the blue domain and the initial value of the green domain to obtain a second residual;

[0059] Calculate the final green domain data according to the initial value of the green domain, the first similarity, and the second similarity;

[0060] Calculate the final red domain data and the final blue domain data through the final green domain data, the first residual, and the second residual;

[0061] Transform the final green domain data, the final blue domain data, and the final red domain data from the Log domain into an RGB image.

[0062] Optionally, in the method for demosaicing an image, after transforming the final green domain data, the final blue domain data, and the final red domain data from the Log domain into an RGB image, it further includes: outputting the RGB image.

[0063] Optionally, in the method for demosaicing an image, the red domain data includes R channel data, the green domain data includes Gr channel data and Gb channel data, and the blue domain data includes B channel data.

[0064] Optionally, in the method for demosaicing an image, the method of mapping the red domain data, the green domain data, and the blue domain data to the Log domain respectively and performing gain processing on the red domain data, the green domain data, and the blue domain data mapped to the Log domain includes:

[0065] Map the R channel data, the Gr channel data, the Gb channel data, and the B channel data to the Log domain respectively to obtain the R channel data, the Gr channel data, the Gb channel data, and the B channel data mapped to the Log domain;

[0066] Perform gain on the R channel data, the Gr channel data, the Gb channel data, and the B channel data mapped to the Log domain.

[0067] Optionally, in the method for demosaicing an image, the method of initializing the green domain data after gain processing to obtain the initial value of the green domain includes:

[0068] Initialize the green domain data after gain processing in the 0° direction to obtain the initial value of the green domain in the 0° direction;

[0069] Initialize the green domain data after gain processing in the 90° direction to obtain the initial value of the green domain in the 90° direction.

[0070] Optionally, in the method for image demosaicing, the method for initializing the red domain data according to the green domain initial value to obtain the red domain initial value includes:

[0071] Extract the initial value in the 0° direction of the green domain in the same row as the R channel and the value of the R channel itself;

[0072] Perform two-dimensional wavelet filtering on the initial value in the 0° direction of the green domain in the same row as the R channel to obtain the green feature component, green H component, green V component, and green D component in the 0° direction, and perform two-dimensional wavelet filtering on the value of the R channel itself to obtain the red feature component in the 0° direction;

[0073] Perform guided filtering on the green feature component to obtain the entire distribution of the red domain feature component in the 0° direction;

[0074] Perform inverse wavelet operation on the green H component, green V component, and green D component in the 0° direction to obtain the red initial value in the 0° direction;

[0075] Extract the initial value in the 90° direction of the green domain in the same column as the R channel and the value of the R channel itself;

[0076] Perform two-dimensional wavelet filtering on the initial value in the 90° direction of the green domain in the same column as the R channel to obtain the green feature component, green H component, green V component, and green D component in the 90° direction, and perform two-dimensional wavelet filtering on the value of the R channel itself to obtain the red feature component in the 90° direction;

[0077] Perform guided filtering on the green feature component in the 90° direction to obtain the entire distribution of the red domain feature component in the 90° direction;

[0078] Perform inverse wavelet operation on the green H component, green V component, and green D component in the 90° direction to obtain the red initial value in the 90° direction.

[0079] Optionally, in the method for image demosaicing, the method for calculating the similarity between the red domain initial value and the green domain initial value to obtain the first similarity includes:

[0080] Calculate the similarity between the red domain and the green domain in the 0° direction;

[0081] Calculate the similarity between the red domain and the green domain in the 90° direction.

[0082] Optionally, in the method for image demosaicing, the method for initializing the blue domain data according to the green domain initial value to obtain the blue domain initial value includes:

[0083] Extract the initial value in the same row as the B channel in the 0° direction of the green domain and the value of the B channel itself;

[0084] Perform two-dimensional wavelet filtering on the initial value in the same row as the R channel in the 0° direction of the green domain to obtain the green feature component, green H component, green V component, and green D component in the 0° direction, and perform two-dimensional wavelet filtering on the value of the B channel itself to obtain the blue feature component in the 0° direction;

[0085] Perform guided filtering on the blue feature component in the 0° direction to obtain the entire distribution of the blue domain feature component in the 0° direction;

[0086] Perform inverse wavelet operation on the green H component, green V component, and green D component in the 0° direction to obtain the blue initial value in the 0° direction;

[0087] Extract the initial value in the same column as the B channel in the 90° direction of the green domain and the value of the B channel itself;

[0088] Perform two-dimensional wavelet filtering on the initial value in the same column as the B channel in the 90° direction of the green domain to obtain the green feature component, green H component, green V component, and green D component in the 90° direction, and perform two-dimensional wavelet filtering on the value of the B channel itself to obtain the blue feature component in the 90° direction;

[0089] Perform guided filtering on the blue feature component in the 90° direction to obtain the entire distribution of the blue domain feature component in the 90° direction;

[0090] Perform guided filtering on the blue feature component in the 90° direction to obtain the entire distribution of the blue domain feature component in the 90° direction.

[0091] Optionally, in the method for image demosaicing, the method for calculating the similarity between the blue domain initial value and the green domain initial value to obtain the second similarity includes:

[0092] Calculate the similarity between the blue domain and the green domain in the 0° direction;

[0093] Calculate the similarity between the blue domain and the green domain in the 90° direction.

[0094] Optionally, in the method for image demosaicing, the method for calculating the final green domain data according to the green domain initial value, the first similarity, and the second similarity includes:

[0095] Calculate the weights of the green domain in the 00°, 90°, 180°, and 270° directions according to the green domain initial value, the similarity between the blue domain and the green domain, and the similarity between the red domain and the green domain;

[0096] Calculate the final green domain data according to the weights of the green domain in the directions of 0°, 90°, 180°, and 270° respectively.

[0097] In the apparatus and method for image demosaicing provided by the present invention, polarization artifacts and jagged phenomena do not occur in complex texture regions, high-contrast regions, and non-directional edge regions of the image, improving the quality of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] Figure 1 is a schematic diagram of an apparatus for image demosaicing according to an embodiment of the present invention;

[0099] Figure 2 is a flowchart of a method for image demosaicing according to an embodiment of the present invention;

[0100] Figures 3 to 45 is a schematic diagram of a data structure for image demosaicing according to an embodiment of the present invention;

[0101] In the figure: 101 - image input unit, 102 - image Log domain transformation unit, 103 - green domain initialization unit, 104 - red domain initialization unit, 105 - red-green similarity calculation unit, 106 - blue domain initialization unit, 107 - blue-green similarity calculation unit, 108 - green domain weighted calculation unit, 109 - red-blue domain weighted calculation unit, 110 - RGB image Log domain inverse transformation unit, 111 - RGB image output unit. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0102] The following will describe the specific embodiments of the present invention in more detail with reference to the schematic diagrams. According to the following description, the advantages and features of the present invention will be clearer. It should be noted that the drawings are all in a very simplified form and use non-precise scales, only for the purpose of facilitating and clearly assisting in explaining the objectives of the embodiments of the present invention.

[0103] In the following text, terms such as "first" and "second" are used to distinguish between similar elements and are not necessarily used to describe a specific order or time sequence. It is understood that, under appropriate circumstances, these terms used in this way can be replaced. Similarly, if the methods described herein include a series of steps, and the order of these steps presented herein is not necessarily the only order in which these steps can be executed, and some of the described steps can be omitted and / or some other steps not described herein can be added to the method.

[0104] Please refer to Figure 1 , the present invention provides an apparatus for image demosaicing, including:

[0105] An image input unit 101 for inputting an image to be processed in Bayer format, where the image to be processed includes red domain data, green domain data, and blue domain data;

[0106] An image Log domain transformation unit 102 for respectively mapping the red domain data, green domain data, and blue domain data to the Log domain, and performing gain processing on the red domain data, green domain data, and blue domain data after being mapped to the Log domain;

[0107] A green domain initialization unit 103 for initializing the gain-processed green domain data to obtain an initial green domain value;

[0108] A red domain initialization unit 104 for initializing the red domain data according to the initial green domain value to obtain an initial red domain value;

[0109] A red-green similarity calculation unit 105 for calculating the similarity between the initial red domain value and the initial green domain value to obtain a first similarity, and at the same time, for calculating the residual between the initial red domain value and the initial green domain value to obtain a first residual;

[0110] A blue domain initialization unit 106 for initializing the blue domain data according to the initial green domain value to obtain an initial blue domain value;

[0111] A blue-green similarity calculation unit 107 for calculating the similarity between the initial blue domain value and the initial green domain value to obtain a second similarity, and at the same time, for calculating the residual between the initial blue domain value and the initial green domain value to obtain a second residual;

[0112] A green domain weighted calculation unit 108 for calculating the final green domain data according to the initial green domain value, the first similarity, and the second similarity;

[0113] A red-blue domain weighted calculation unit 109 for calculating the final red domain data and the final blue domain data through the final green domain data, the first residual, and the second residual;

[0114] An RGB image Log domain inverse transformation unit 110 for transforming the final green domain data, the final blue domain data, and the final red domain data from the Log domain into an RGB image.

[0115] Further, it further includes: an RGB image output unit 111 for outputting an RGB image. The input image to be processed is in Bayer format, which is then converted to the log domain. On the log domain, joint wavelet decomposition and guided filtering are performed to obtain the similarity information of each channel in different directions, and the demosaicing result is adaptively calculated, effectively reducing the polarization artifacts and jaggedness in the complex texture area, high-contrast area, and non-directional edge area of the image, and improving the authenticity of the image. When finally outputting the image, it needs to be converted to RGB format.

[0116] Further, the red domain data includes R-channel data, the green domain data includes Gr-channel data and Gb-channel data, and the blue domain data includes B-channel data.

[0117] Preferably, the image Log domain transformation unit 102 includes:

[0118] A Log domain mapping module for respectively mapping the R-channel data, Gr-channel data, Gb-channel data, and B-channel data to the Log domain to obtain the R-channel data, Gr-channel data, Gb-channel data, and B-channel data mapped to the Log domain;

[0119] A channel data gain compensation module for performing gain on the R-channel data, Gr-channel data, Gb-channel data, and B-channel data mapped to the Log domain.

[0120] Preferably, the green domain initialization unit 103 includes:

[0121] A green domain 0° direction initialization module for initializing the gain-processed green domain data in the 0° direction to obtain the initial value in the 0° direction of the green domain;

[0122] A green domain 90° direction initialization module for initializing the gain-processed green domain data in the 90° direction to obtain the initial value in the 90° direction of the green domain.

[0123] Preferably, the red domain initialization unit 104 includes:

[0124] A first 0° direction value extraction module for extracting the initial value in the 0° direction of the green domain in the same row as the R channel and the value of the R channel itself;

[0125] A first two-dimensional wavelet filtering module for performing two-dimensional wavelet filtering on the initial value in the 0° direction of the green domain in the same row as the R channel to obtain the green feature component, green H component, green V component, and green D component in the 0° direction, and for performing two-dimensional wavelet filtering on the value of the R channel itself to obtain the red feature component in the 0° direction;

[0126] The first red domain reference guided filtering module is used to perform guided filtering on the green feature component to obtain the entire distribution of the red domain feature component in the 0° direction;

[0127] The first two-dimensional wavelet synthesis module is used to perform inverse wavelet operations on the green H component, green V component, and green D component in the 0° direction to obtain the initial red value in the 0° direction;

[0128] The first 90° direction numerical extraction module is used to extract the initial value in the same column as the R channel in the 90° direction of the green domain and the value of the R channel itself;

[0129] The second two-dimensional wavelet filtering module is used to perform two-dimensional wavelet filtering on the initial value in the same column as the R channel in the 90° direction of the green domain to obtain the green feature component, green H component, green V component, and green D component in the 90° direction, and is used to perform two-dimensional wavelet filtering on the value of the R channel itself to obtain the red feature component in the 90° direction;

[0130] The second red domain reference guided filtering module is used to perform guided filtering on the green feature component in the 90° direction to obtain the entire distribution of the red domain feature component in the 90° direction;

[0131] The second two-dimensional wavelet synthesis module is used to perform inverse wavelet operations on the green H component, green V component, and green D component in the 90° direction to obtain the initial red value in the 90° direction.

[0132] Preferably, the red-green similarity calculation unit 105 includes:

[0133] The first 0° direction similarity calculation module is used to calculate the similarity between the red domain and the green domain in the 0° direction;

[0134] The first 90° direction similarity calculation module is used to calculate the similarity between the red domain and the green domain in the 90° direction.

[0135] Preferably, the blue domain initialization unit 106 includes:

[0136] The second 0° direction numerical extraction module is used to extract the initial value in the same row as the B channel in the 0° direction of the green domain and the value of the B channel itself;

[0137] The third two-dimensional wavelet filtering module is used to perform two-dimensional wavelet filtering on the initial value in the same row as the R channel in the 0° direction of the green domain to obtain the green feature component, green H component, green V component, and green D component in the 0° direction, and is used to perform two-dimensional wavelet filtering on the value of the B channel itself to obtain the blue feature component in the 0° direction;

[0138] The first blue domain reference guided filtering module is used to perform guided filtering on the blue feature component in the 0° direction to obtain the entire distribution of the blue domain feature component in the 0° direction;

[0139] The third two-dimensional wavelet synthesis module is used to perform inverse wavelet operations on the green H component, green V component, and green D component in the 0° direction to obtain the initial blue value in the 0° direction;

[0140] The second 90° direction numerical extraction module is used to extract the initial value in the same column as the B channel in the 90° direction of the green domain and the value of the B channel itself;

[0141] The fourth two-dimensional wavelet filtering module is used to perform two-dimensional wavelet filtering on the initial value in the same column as the B channel in the 90° direction of the green domain to obtain the green feature component, green H component, green V component, and green D component in the 90° direction, and is used to perform two-dimensional wavelet filtering on the value of the B channel itself to obtain the blue feature component in the 90° direction;

[0142] The second blue domain reference guided filtering module is used to perform guided filtering on the blue feature component in the 90° direction to obtain the entire distribution of the blue domain feature component in the 90° direction;

[0143] The fourth two-dimensional wavelet synthesis module is used to perform guided filtering on the blue feature component in the 90° direction to obtain the entire distribution of the blue domain feature component in the 90° direction.

[0144] Preferably, the blue-green similarity calculation unit 107 includes:

[0145] The second 0° direction similarity calculation module is used to calculate the similarity between the blue domain and the green domain in the 0° direction;

[0146] The second 90° direction similarity calculation module is used to calculate the similarity between the blue domain and the green domain in the 90° direction.

[0147] Preferably, the green domain weighted calculation unit 108 includes:

[0148] The weight calculation module is used to calculate the weights of the green domain in the 00°, 90°, 180°, and 270° directions respectively according to the green domain initial value, the similarity between the blue domain and the green domain, and the similarity between the red domain and the green domain;

[0149] The weighted calculation module is used to calculate the final green domain data according to the weights of the green domain in the 00°, 90°, 180°, and 270° directions respectively.

[0150] Please refer to Figure 2 , the present invention provides an image demosaicing method using an image demosaicing device, including:

[0151] S1: Input the image to be processed in Bayer format, where the image to be processed includes red domain data, green domain data, and blue domain data;

[0152] S2: Map the red domain data, green domain data, and blue domain data to the Log domain respectively, and perform gain processing on the red domain data, green domain data, and blue domain data mapped to the Log domain;

[0153] S3: Initialize the green domain data after gain processing to obtain the initial value of the green domain;

[0154] S4: Initialize the red domain data according to the initial value of the green domain to obtain the initial value of the red domain;

[0155] S5: Calculate the similarity between the initial value of the red domain and the initial value of the green domain to obtain the first similarity. At the same time, calculate the residual between the initial value of the red domain and the initial value of the green domain to obtain the first residual;

[0156] S6: Initialize the blue domain data according to the initial value of the green domain to obtain the initial value of the blue domain;

[0157] S7: Calculate the similarity between the initial value of the blue domain and the initial value of the green domain to obtain the second similarity. At the same time, it is also used to calculate the residual between the initial value of the blue domain and the initial value of the green domain to obtain the second residual;

[0158] S8: Calculate the final green domain data according to the initial value of the green domain, the first similarity, and the second similarity;

[0159] S9: Calculate the final red domain data and the final blue domain data through the final green domain data, the first residual, and the second residual;

[0160] S10: Transform the final green domain data, the final blue domain data, and the final red domain data from the Log domain into an RGB image.

[0161] First, execute step S1. The image data to be processed input by the image data input unit is in Bayer mode. According to different configurations, the bit width of the input image data is also different, and the default is 8-bit input image data. The Bayer image data consists of four channels, where red occupies one channel, namely the R channel, green occupies two channels, namely the Gr and Gb channels, and blue occupies one channel, namely the B channel. Therefore, the red domain data includes R channel data, the green domain data includes Gr channel data and Gb channel data, and the blue domain data includes B channel data. The Bayer image data has 4 formats according to 4 different arrangements of the four channels. For example, Figure 3It is a Bayer image in the R, Gr, Gb, B format. Of course, there are also Bayer images in the Gr, R, B, Gb format, Bayer images in the B, Gb, Gr, R format, and Bayer images in the Gb, B, R, Gr format.

[0162] Next, step S2 is executed. The image Log domain transformation unit maps the red domain data, green domain data, and blue domain data to the Log domain respectively, so that the contrast of the high-contrast region of the image to be processed is compressed to reduce the phenomena of polarized false colors and jagged edges in the high-contrast region; and the red domain data, green domain data, and blue domain data after being mapped to the Log domain are subjected to gain processing to reduce the interference and influence of uncorrected chromaticity on the image demosaicing process. The specific method is step S21 and step S22. First, step S21 is executed: The Log domain mapping module maps the R channel data, Gr channel data, Gb channel data, and B channel data to the Log domain respectively to obtain the R channel data, Gr channel data, Gb channel data, and B channel data after being mapped to the Log domain. The calculation formula is as follows:

[0163]

[0164] where, log n (Bayer) is the base of the logarithmic function log, and n can take different values according to images with different bit widths. Here, by default, n = 2. The maximum value of LogRaw is 2^LFShift - 1.

[0165] Next, step S22 is executed: The channel data gain compensation module performs gain on the R channel data, Gr channel data, Gb channel data, and B channel data after being mapped to the Log domain. The data gain compensation for each channel is obtained by combining the configured gain values to get LogRaw. The calculation formula is as follows:

[0166]

[0167] where, LogRaw is the image after gain, LogRaw_R is the R channel data before gain, LogRaw_Gr is the Gr channel data before gain, LogRaw_B is the B channel data before gain, LogRaw_Gb is the Gb channel data before gain, Gain_R is the data gain value of the R channel, Gain_Gr is the data gain value of the Gr channel, Gain_B is the data gain value of the B channel, and Gain_Gb is the data gain value of the Gb channel.

[0168] Since green occupies two channels out of the four channels of Bayer image data, the vast majority of the reference for determining the image brightness comes from the green domain. Therefore, it is possible to preferentially initialize the green domain, and use the initial value of the green domain as the reference brightness to more accurately guide the actual values of each channel. Therefore, after step S2 is executed, step S3 is then executed to initialize the green domain data after gain processing to obtain the initial value of the green domain. The method of step S3 includes step S31 and step S32.

[0169] First, execute step S31: The green domain 0° direction initialization module initializes the green domain data after gain processing in the 0° direction to obtain the initial value of the green domain in the 0° direction; the data arrangement of the 5x5 window centered on the current pixel point is as Figure 4 , then there are two cases for calculating the initial value of the green domain in the 0° direction according to the channel where the current point is located.

[0170] If the channel where the current point is located is the G channel, including the Gr channel and the Gb channel, the calculation formula for the initial value of the green domain in the 0° direction is as follows:

[0171] Est000_G = P13;

[0172] Where: Est000_G is the initial value of the green domain in the 0° direction, and P13 is the data of P13 in the 5x5 window data.

[0173] If the channel where the current point is located is the R channel or the B channel, the calculation formula for the initial value of the green domain in the 0° direction is as follows:

[0174]

[0175] Where: Est000_G is the initial value of the green domain in the 0° direction, and P11, P12, P13, P14, and P15 are all data in the 5x5 window data.

[0176] Next, execute step S32: The green domain 90° direction initialization module initializes the green domain data after gain processing in the 90° direction to obtain the initial value of the green domain in the 90° direction. Still taking Figure 4 as an example.

[0177] If the channel where the current point is located is the G channel, the formula for calculating the initial value of the green domain in the 90° direction according to the channel where the current point is located is as follows:

[0178] Est090_G = P13;

[0179] Where: Est090_G is the initial value of the green domain in the 90° direction, and P11, P12, P13, P14, and P15 are all data in the 5x5 window data.

[0180] If the current point is in the R channel or the B channel, the formula for calculating the initial value in the 90° direction of the green region is as follows:

[0181]

[0182] Where: Est090_G is the initial value in the 90° direction of the green region, and P3, P8, P13, P18, and P23 are all data in the 5x5 window data.

[0183] Next, perform step S4 to initialize the red region data based on the initial value of the green region to obtain the initial value of the red region. The specific method is divided into steps S41 to S48.

[0184] Since one of the four channels of the Bayer image data is red, and in the 0° direction, the R channel and the B channel are not related to each other. Therefore, extract the initial value in the 0° direction of the green region that is in the same row as the R channel, and calculate it with the R channel itself. Therefore, first perform step S41: Extract the initial value in the 0° direction of the green region that is in the same row as the R channel and the value of the R channel itself. The process is as Figures 5 to 8 shown, Figure 5 The marked is the arrangement of the 5x5 window data centered on the R channel, Figure 6 is Figure 5 the coordinate index, Figure 7 and Figure 8 are the initial value in the 0° direction of the green region that is in the same row as the R channel and the value of the R channel itself after extraction. Where E represents Est000_G, and R represents the corresponding value of the R channel. They are respectively named Est000_G2 and Raw000_R2. It can be seen that after this extraction, the format of Est000_G2 and Raw000_R2 becomes half of the original format. That is, if the original format is 1920x1080, the extracted format is 1920x540.

[0185] Next, perform step S42: Perform two-dimensional wavelet filtering on the initial value in the 0° direction of the green region that is in the same row as the R channel to obtain the green feature component, green H component, green V component, and green D component in the 0° direction, and perform two-dimensional wavelet filtering on the value of the R channel itself to obtain the red feature component in the 0° direction; perform two-dimensional wavelet filtering calculations with a scale of 1 on the obtained Est000_G2 and Raw000_R2 respectively. Two-dimensional wavelet filtering is a mature frequency domain transformation method in mathematics, which is used to retain the features of the image data, and the remaining information is retained in the H component, V component, and D component. The calculation formula is as follows:

[0186] [cA_Est000_G2, cH, cV, cD] = stw2(Est000_G2, level);

[0187] [cA_Raw000_R2, ~, ~, ~] = swt2(Raw000_R2, level);

[0188] Among them, cA_Est000_G2 is the wavelet feature component of Est000_G2, cH, cV, and cD are the H component, V component, and D component of Est000_G2 respectively, level is the demarcation scale, and here level = 1. cA_Raw000_R2 is the wavelet feature component of Raw000_R2.

[0189] Next, perform step S43: Perform guided filtering on the green feature component to obtain the entire distribution in the 0° direction of the red domain feature component; Guided filtering is a process of linear transformation, and its input is composed of the input image and the reference image. After wavelet filtering calculation, the green domain feature component and the red domain feature component are consistent. Therefore, the red domain feature component is used as the reference image, and the green domain feature component is used as the input image. In this way, after guided filtering, the entire distribution in the 0° direction of the red domain feature component can be obtained. The calculation formula is as follows:

[0190] cA_Est000_R2 = guidedfilter(cA_Est000_G2, cA_Raw000_R2 × 2, r, eps);

[0191] Among them, cA_Est000_R2 is the wavelet feature component of Est000_R2, cA_Est000_G2 is the wavelet feature component of Est000_G2, cA_Raw000_R2 is the wavelet feature component of Raw000_R2, r is the radius of the guided filtering window, and here the default value r = 1 is selected. eps is a parameter to prevent division by zero and control the intensity of guided filtering, and the default value is 1. guidedfilter() is the guided filtering. Since half of the values of Raw000_R2 are 0, the overall intensity of its feature distribution is half of the original. Therefore, cA_Raw000_R2 needs to be processed with a 2-fold gain.

[0192] Next, perform step S44: Perform inverse wavelet transformation on the green H component, green V component, and green D component in the 0° direction to obtain the red initial value in the 0° direction; After obtaining the wavelet feature component cA_Est000_R2, perform inverse wavelet transformation, that is, wavelet synthesis, according to the information of cH, cV, and cD, and the red initial value in the 0° direction can be obtained. The calculation formula is as follows:

[0193] Est000_R2 = iswt2(cA_Est000_R2, cH, cV, cD);

[0194] Among them, Est000_R2 is the initial red value in the 0° direction, iswt2() is the inverse wavelet transform, cA_Est000_R2 is the wavelet feature component of Est000_R2, and cH, cV, and cD are the H component, V component, and D component of Est000_G2. Similar to Est000_G2, Est000_R2 is an image with a size half of the original image size. Restoring it to the original image size in the 0° direction gives Est000_R. This process is as shown in Figure 9 and Figure 10 shown.

[0195] Among them, if the channel where the current point R13 is located is the R channel, the formula for calculating Est000_R is as follows:

[0196] Est000_R = P13;

[0197] Among them, Est000_R is the initial red value in the 0° direction of the original image size. P13 is the value corresponding to the current point in the R channel in Figure 5 the above.

[0198] Next, perform step S45: Extract the initial value in the 90° direction of the green domain in the same column as the R channel and the value of the R channel itself. First, extract the initial value in the 90° direction of the green domain in the same column as the R channel. This process is as shown in Figure 5 , Figure 6 , Figure 11 and Figure 12 shown. Figure 5 and Figure 6 marked are the 5x5 window data arrangement centered on the R channel and its coordinate indices, Figure 11 and Figure 12 are the initial value in the 90° direction of the green domain in the same column as the R channel and the value of the R channel itself after extraction. Among them, E represents the initial value Est090_G in the 90° direction of the green domain, and R represents the corresponding value of the R channel. They are respectively named Est090_G2 and Raw090_R2. It can be seen that after this extraction, the image sizes of Est090_G2 and Raw090_R2 become half of the original image size. That is, if the original image size is 1920x1080, the image size after extraction is 960x1080.

[0199] Next, perform step S46: Perform two-dimensional wavelet filtering on the initial values in the same column as the R channel in the 90° direction of the green domain to obtain the green feature component, green H component, green V component, and green D component in the 90° direction, and perform two-dimensional wavelet filtering on the original value of the R channel itself to obtain the red feature component in the 90° direction; perform two-dimensional wavelet filtering calculations with a scale of 1 on the obtained Est090_G2 and Raw090_R2 respectively. Two-dimensional wavelet filtering is a mature frequency-domain transformation method in the mathematical sense, used to retain the features of the image data, and the remaining information is retained in the H component, V component, and D component. The calculation formula is as follows:

[0200] [cA_Est090_G2, cH, cV, cD] = swt2(Est090_G2, level);

[0201] [cA_Raw090_R2, ~, ~, ~] = swt2(Raw090-R2, level);

[0202] Among them, cA_Est090_G2 is the wavelet feature component of Est090_G2, cH, cV, and cD are its corresponding H component, V component, and D component, level is the demarcation scale, and here level = 1. cA_Raw090_R2 is the wavelet feature component of Raw090_R2.

[0203] Next, perform step S47: Perform guided filtering on the green feature component in the 90° direction to obtain the entire distribution of the red domain feature component in the 90° direction; Guided filtering is a process of linear transformation, and its input is composed of the input image and the reference image. After wavelet filtering calculation, the green domain feature component and the red domain feature component are consistent. Therefore, the red domain feature component is used as the reference image, and the green domain feature component is used as the input image. In this way, after guided filtering, the entire distribution of the red domain feature component in the 90° direction can be obtained. The calculation formula is as follows:

[0204] cA_Est090_R2 = guidedfilter(cA_Est090_G2, cA_Raw090_R2×2, r, eps);

[0205] Among them, since half of the values of Raw090_R2 are 0, the overall intensity of its feature distribution is half of the original. Therefore, cA_Raw090_R2 needs to be processed with a 2-fold gain; r is the radius of the guided filtering window, and the default value r = 1 is selected here. eps is a parameter to prevent division by zero and control the intensity of guided filtering. The default value is 1.

[0206] Next, step S48 is executed: the green H component, green V component and green D component in the 90° direction are subjected to wavelet inverse operation to obtain the initial red value in the 90° direction. After obtaining the wavelet characteristic component cA_Est090_R2, the wavelet inverse operation is performed according to the information of cH, cV and cD, i.e. wavelet synthesis, to obtain the initial red value in the 90° direction. The calculation formula is as follows:

[0207] Est090_R2=iswt2(cA_Est090_R2, cH, cV, cD);

[0208] Est090_R2 is the same as Est090_G2, which is an image with half the size of the original image. It is restored to the original size at 90° to obtain Est090_R. The process is as follows: Figure 13 and Figure 14 shown.

[0209] Figure 13 is with Figures 11 to 12 The initial value of the red domain corresponding to the matching pixel, Figure 13 The R in the figure represents Est090_R2. When it is restored to the original format, the mapping effect is as follows: Figure 14 shown.

[0210] If the channel where the current point R13 is located is the R channel, then:

[0211] EST090_R=P13;

[0212] Among them, Est090_R is the initial red value in the 90° direction of the original image. P13 is the value corresponding to the R channel at the current point. Figure 5 The value in .

[0213] Next, step S5 is executed to calculate the similarity between the red domain initial value and the green domain initial value to obtain a first similarity, and at the same time, calculate the residual between the red domain initial value and the green domain initial value to obtain a first residual. Step S5 includes steps S51 to S52.

[0214] Step S51: Calculate the similarity between the red domain and the green domain in the 0° direction; specifically, Figure 15 and Figure 16 The initial values ​​of the green domain 0° direction and the red domain 0° direction with the R channel as the center point are shown respectively. If the channel where the current point is located is the R channel, taking R13 as an example, the calculation formula of the first residual is as follows:

[0215]

[0216] Among them, Delta000_GR(13) is the first residual data corresponding to index 13, E13 is the initial value of the green domain in the 0° direction corresponding to index 13, R13 is the initial value of the red domain in the 0° direction corresponding to index 13, R12 is the initial value of the red domain in the 0° direction corresponding to index 12, and R14 is the initial value of the red domain in the 0° direction corresponding to index 14.

[0217] If the channel where the current point is located is the G channel, taking E12 and E14 as examples, the calculation formula for the similarity between the red domain and the green domain in the 0° direction is as follows:

[0218] Delta000_GR(12) = E12 - R12;

[0219] Delta000_GR(14) = E14 - R14;

[0220] Among them, Delta000_GR(12) is the first residual data corresponding to index 12, Delta000_GR(14) is the first residual data corresponding to index 14, E12 is the initial value of the green domain in the 0° direction corresponding to index 12, E14 is the initial value of the green domain in the 0° direction corresponding to index 14, R12 is the initial value of the red domain in the 0° direction corresponding to index 12, and R14 is the initial value of the red domain in the 0° direction corresponding to index 14.

[0221] Compare and calculate the residuals on both sides of the center point to obtain the corresponding similarity in the direction extension. The formula for calculating the similarity between the red domain and the green domain in the 0° direction is as follows:

[0222] Sim000_GR(13) = Delta000_GR(12) - Delta000_GR(14);

[0223] Among them, Sim000_GR(13) is the red-green similarity in the 0° direction corresponding to index 13.

[0224] Delta000_GR(12) is the first residual data corresponding to index 12. Delta000_GR(14) is the first residual data corresponding to index 14

[0225] Step S52: Calculate the similarity between the red domain and the green domain in the 90° direction. Figure 17 and Figure 18 respectively show the initial values of the green domain in the 90° direction and the initial values of the red domain in the 90° direction with the R channel as the center point. We first perform residual calculation based on the information of both.

[0226] If the channel where the current point is located is the R channel, taking R13 as an example, the calculation formula for this process is as follows:

[0227]

[0228] If the channel where the current point is located is the G channel, taking E8 and E18 as examples, the calculation formula is as follows:

[0229] Delta090_GR(8) = E8 - R8;

[0230] Delta090_GR(18) = E18 - R18;

[0231] The calculation formula for the similarity between the red domain and the green domain in the 90° direction is as follows:

[0232] Sim090_GR(13) = Delta090 - GR(8) - Delta000 - GR(18);

[0233] Next, execute step S6 to initialize the blue domain data according to the initial value of the green domain to obtain the initial value of the blue domain. The method of step S6 includes steps S61 to S68.

[0234] Step S61: Extract the initial value of the same row as the B channel in the 0° direction of the green domain and the value of the B channel itself; Since blue occupies one channel among the four channels of the Bayer image data, and in the 0° direction, the B channel and the R channel are not related to each other, therefore, extract the initial value of the same row as the B channel in the 0° direction of the green domain and the B channel itself for calculation.

[0235] First, extract the initial value of the same row as the B channel in the 0° direction of the green domain. This process is as Figure 19 , Figure 20 , Figure 21 shown.

[0236] Figure 19 and Figure 20 marked are the 5x5 window data arrangement centered on the B channel and its coordinate index, Figure 21 is the initial value of the same row as the B channel in the 0° direction of the green domain and the value of the B channel itself after extraction. Among them, E represents Est000_G, B represents the corresponding value of the B channel, and they are named Est000_G2 and Raw000_B2 respectively. It can be seen that after this extraction, the format of Est000_G2 and Raw000_B2 becomes half of the original format, that is, if the original format is 1920x1080, the extracted format is 1920x540.

[0237] S62: Perform two-dimensional wavelet filtering on the initial values in the 0° direction of the green domain in the same row as the R channel to obtain the green feature component, green H component, green V component, and green D component in the 0° direction, and perform two-dimensional wavelet filtering on the original value of the B channel to obtain the blue feature component in the 0° direction; perform two-dimensional wavelet filtering calculations on the obtained Est000_G2 and Raw000_B2 with a scale of 1 respectively. Two-dimensional wavelet filtering is a mature frequency-domain transformation method in the mathematical sense, used to retain the features of image data, and the remaining information is retained in the H component, V component, and D component. The formula for this calculation process is as follows:

[0238] [cA_Est000_G2, cH, cV, cD] = swt2(Est000_G2, level);

[0239] [cA_Raw000_B2, ~, ~, ~] = swt2(Raw000 - B2, level);

[0240] Among them, cA_Est000_G2 is the wavelet feature component of Est000_G2, cH, cV, and cD are its corresponding H component, V component, and D component respectively, level is the boundary scale, and here level = 1. cA_Raw000_B2 is the wavelet feature component of Raw000_B2.

[0241] S63: Perform guided filtering on the blue feature component in the 0° direction to obtain the entire distribution of the blue domain feature component in the 0° direction; guided filtering is a process of linear transformation, and its input is composed of the input image and the reference image. After wavelet filtering calculation, the green domain feature component and the blue domain feature component have consistency. Therefore, the blue domain feature component is used as the reference image, and the green domain feature component is used as the input image. In this way, after guided filtering, the entire distribution of the blue domain feature component in the 0° direction can be obtained. The formula for this calculation process is as follows:

[0242] cA_Est000_B2 = guidedfilter(cA_Est000_G2, cA_Raw000_B2 × 2, r, eps);

[0243] Among them, since half of the values of Raw000_B2 are 0, the overall intensity of its feature distribution is half of the original. Therefore, cA_Raw000_B2 needs to be processed with a 2-fold gain; r is the radius of the guided filtering window, and the default value r = 1 is selected here. eps is a parameter to prevent division by zero and control the intensity of guided filtering, and the default value is 1.

[0244] S64: Perform inverse wavelet transform on the green H component, green V component, and green D component in the 0° direction to obtain the initial blue value in the 0° direction. After obtaining the wavelet feature component cA_Est000_B2, perform inverse wavelet transform, that is, wavelet synthesis, based on the information of cH, cV, and cD, and the initial blue value in the 0° direction can be obtained. The calculation formula is as follows:

[0245] Est000_B2 = iswt2(cA_Est000_B2, cH, cV, cD);

[0246] Est000_B2 is the initial blue value in the 0° direction, and Est000_G2 is the initial green value in the 0° direction. Similar to Est000_G2, it is an image with a size half of the original image. Restore it to the original image size in the 0° direction to obtain Est000_B. This process is as Figure 22 and Figure 23 shown.

[0247] Figure 22 is the initial green domain value corresponding to the pixel points that match Figure 21 . The B in Figure 22 represents Est000_B2. Restore it to the original image size, and its mapping effect is as Figure 23 shown.

[0248] Among them, if the channel where the current point B13 is located is the B channel, then:

[0249] Est000_B = P13

[0250] Among them, Est000_B is the initial blue value in the 0° direction of the original image, and P13 is the value corresponding to the current point in the B channel Figure 5 in.

[0251] S65: Extract the initial value in the 90° direction of the green domain in the same column as the B channel and the value of the B channel itself; Since the blue color occupies one channel among the four channels of the Bayer image data, and in the 90° direction, the B channel and the R channel are not related to each other, therefore, we extract the initial value in the 90° direction of the green domain in the same column as the B channel and the B channel itself for calculation.

[0252] First, extract the initial value in the 90° direction of the green domain in the same column as the B channel. This process is as Figure 19 and Figure 20 and Figure 24 and Figure 25 shown.

[0253] Figure 19 and Figure 20 ​The marked data arrangement of the 5x5 window centered on the B channel and its coordinate index Figure 24 and Figure 25 are respectively the initial value in the same column as the B channel in the 90° direction of the extracted green domain and the value of the B channel itself. Among them, E represents Est090_G, B represents the corresponding value of the B channel, and the two are respectively named Est090_G2 and Raw090_B2. It can be seen that after this extraction, the amplitude of Est090_G2 and Raw090_B2 becomes half of the original amplitude. That is, if the original amplitude is 1920x1080, the amplitude after extraction is 960x1080.

[0254] S66: Perform two-dimensional wavelet filtering on the initial value in the 90° direction of the green domain in the same column as the B channel to obtain the green feature component, green H component, green V component, and green D component in the 90° direction, and perform two-dimensional wavelet filtering on the value of the B channel itself to obtain the blue feature component in the 90° direction; perform two-dimensional wavelet filtering calculations with a scale of 1 on the obtained Est090_G2 and Raw090_B2 respectively. Two-dimensional wavelet filtering is a mature frequency-domain transformation method in the mathematical sense, used to retain the features of image data, and the remaining information is retained in the H component, V component, and D component. The formula for this calculation process is as follows:

[0255] [cA_Est090_G2, cH, cV, cD] = swt2(Est090_G2, level);

[0256] [cA_Raw090_B2, ~, ~, ~] = swt2(Raw090_B2, level);

[0257] Among them, cA_Est090_G2 is the wavelet feature component of Est090_G2, cH, cV, and cD are its corresponding H component, V component, and D component, level is the boundary scale, and here level = 1. cA_Raw090_B2 is the wavelet feature component of Raw090_B2.

[0258] S67: Perform guided filtering on the blue feature component in the 90° direction to obtain the entire distribution of the blue domain feature component in the 90° direction; guided filtering is a process of linear transformation, and its input consists of the input image and the reference image. After the wavelet filtering calculation, the green domain feature component and the blue domain feature component are consistent. Therefore, the blue domain feature component is used as the reference image, and the green domain feature component is used as the input image. In this way, after guided filtering, the entire distribution of the blue domain feature component in the 90° direction can be obtained. The calculation formula is as follows:

[0259] cA_Est090_B2 = guidedfilter(cA_Est090_G2, cA_Raw090_B2 * 2, r, eps);

[0260] Among them, since half of the values of Raw090_B2 are 0, the overall intensity in the feature distribution is half of the original. Therefore, cA_Raw090_B2 needs to be processed with a 2-fold gain; r is the radius of the guided filter window. Here, the default value r = 1 is selected. eps is a parameter to prevent division by zero and control the intensity of the guided filter. The default value is 1.

[0261] S68: Perform guided filtering on the blue feature component in the 90° direction to obtain the entire distribution of the blue domain feature component in the 90° direction. After obtaining the wavelet feature component cA_Est090_B2, based on the information of cH, cV, and cD, perform inverse wavelet transformation, that is, wavelet synthesis, to obtain the initial red value in the 90° direction. The calculation formula is as follows:

[0262] Est090_B2 = iswt2(cA_Est090_B2, cH, cV, cD);

[0263] Similar to Est090_G2, Est090_B2 is an image with half of the original image size. Restore it to the original image size in the 90° direction to obtain Est090_B. The process is as Figure 26 and Figure 27 shown:

[0264] Figure 26 is the initial value of the blue domain corresponding to the pixel points matching Figure 24 and Figure 25 . The B in Figure 26 represents Est090_B2. Restore it to the original image size, and its mapping effect is as Figure 27 shown.

[0265] Among them, if the channel where the current point R13 is located is the B channel, then:

[0266] Est090_B = P13

[0267] Among them, Est090_B is the initial blue value in the 90° direction of the original image size, and P13 is the value corresponding to the current point being in the B channel in Figure 5 .

[0268] Next, execute step S7 to calculate the similarity between the initial value of the blue domain and the initial value of the green domain to obtain the second similarity. At the same time, it is also used to calculate the residual between the initial value of the blue domain and the initial value of the green domain to obtain the second residual:

[0269] S71: Calculate the similarity between the blue region and the green region in the 0° direction; Figure 28 and Figure 29 respectively show the initial values of the green region in the 0° direction and the initial values of the blue region in the 0° direction with the R channel as the center point. First, perform residual calculation based on the information of both to obtain the second residual.

[0270] If the channel where the current point is located is the B channel, taking B13 as an example, the calculation formula for the second residual is as follows:

[0271]

[0272] where Delta000_GB(13) is the second residual.

[0273] If the channel where the current point is located is the G channel, taking E12 and E14 as an example, the calculation formula for the second residual is as follows:

[0274] Delta000_GB(12) = E12 - B12;

[0275] Delta000_GB(14) = E14 - B14;

[0276] Compare and calculate the residuals on both sides of the center point to obtain the corresponding similarity in the directionality. Then, the calculation formula for the second similarity is as follows:

[0277] Sim000_GB(13) = Delta000_GB(12) - Delta000_GB(14);

[0278] S72: Calculate the similarity between the blue region and the green region in the 90° direction. Figure 30 and Figure 31 respectively show the initial values of the green region in the 90° direction and the initial values of the blue region in the 90° direction with the R channel as the center point. First, perform residual calculation based on the information of both.

[0279] If the channel where the current point is located is the B channel, taking B13 as an example, the process formula for calculating the residual is as follows:

[0280]

[0281] If the channel where the current point is located is the G channel, taking E8 and E18 as an example, the calculation formula for the residual is as follows:

[0282] Delta090_GB(8) = E8 - B8;

[0283] Delta090_GB(18) = E18 - B18;

[0284] Compare and calculate the residuals on both sides of the center point to obtain the corresponding similarity in the direction extension. The calculation formula is as follows:

[0285] Sim090_GB(13) = Delta090_GB(8) - Delta000_GB(18);

[0286] Next, execute step S8. The method for calculating the final green domain data based on the initial value of the green domain, the first similarity, and the second similarity includes:

[0287] S81: Calculate the weights of the green domain in the directions of 0°, 90°, 180°, and 270° according to the initial value of the green domain, the similarity between the blue domain and the green domain, and the similarity between the red domain and the green domain; Since the similarity between red and green and the similarity between blue and green are independent in the 0° direction, and the similarity between red and green and the similarity between blue and green are independent in the 90° direction, the similarity between red and green and the similarity between blue and green can be combined, as Figure 32 and Figure 33 shown.

[0288] Then, perform filtering calculations on Sim000 in the 0° direction and the 180° direction respectively, and perform filtering calculations on Sim090 in the 90° direction and the 270° direction respectively.

[0289] The filtering kernels kernel000 and kernel180 in the 0° direction and the 180° direction are respectively as Figure 34 and Figure 35 shown.

[0290] The filtering kernels kernel090 and kernel270 in the 90° direction and the 270° direction are as Figure 36 and Figure 37 shown.

[0291] The weight calculation formula is as follows:

[0292]

[0293]

[0294]

[0295]

[0296] Among them, wgt000, wgt180, wgt090, and wgt270 are the weights in the 00°, 90°, 180°, and 270° directions respectively, kernel000, kernel180, kernel090, and kernel270 are the filter kernels in the 00°, 90°, 180°, and 270° directions respectively, and sim000, sim180, sim090, and sim270 are the second similarities in the 00°, 90°, 180°, and 270° directions respectively.

[0297] S82: Calculate the final green domain data according to the weights of the green domain in the 00°, 90°, 180°, and 270° directions respectively. Since the red-green and blue-green residuals are independent in the 0° direction and the red-green and blue-green residuals are independent in the 90° direction, the red-green and blue-green residuals can be combined, as shown in Figure 38 and Figure 39 respectively.

[0298] Then, perform filtering calculations on Delta000 in the 0° and 180° directions respectively, and perform filtering calculations on Delta090 in the 90° and 270° directions respectively. The filter kernels kernel000 and kernel180 in the 0° and 180° directions are as shown in Figure 40 and Figure 41 respectively. The filter kernels kernel090 and kernel270 in the 90° and 270° directions are as shown in Figure 42 and Figure 43 respectively.

[0299] The calculation formulas for the filtering results in the corresponding directions are as follows:

[0300]

[0301]

[0302]

[0303]

[0304] Among them, DeltaF000, DeltaF180, DeltaF090, and DeltaF270 are the filtering results in the 00°, 90°, 180°, and 270° directions respectively.

[0305] The final weighted calculation formula is as follows:

[0306]

[0307] The calculation formula for the final result of the green domain is as follows:

[0308]

[0309] Next, the method for calculating the final red domain data and the final blue domain data through the final green domain data, the first residual, and the second residual includes:

[0310] First, perform a filtering process on the final residual, and the filtering kernel kernel0 is as Figure 44 shown. The calculation formula is as follows:

[0311]

[0312] Then, perform a filtering process on the green domain value and RGB_R0, and the filtering kernel kernel1 is as Figure 45 shown. The calculation formula for this process is as follows:

[0313]

[0314]

[0315] Equivalently, we can obtain the final result of the blue domain. The calculation formula for this process is as follows:

[0316]

[0317]

[0318]

[0319] Perform the last step. After transforming the final green domain data, the final blue domain data, and the final red domain data from the Log domain into an RGB image, it further includes: outputting the RGB image.

[0320] According to the obtained RGB data, perform an inverse Log domain mapping on it separately for the R / G / B channels. The inverse Log domain mapping calculation formula is as follows:

[0321]

[0322] RGB_Out = (RGBO) n ;

[0323] where LFShift is the corresponding number of bits in , and n is the base of the logarithmic function log in the formula

[0324] In summary, in the apparatus and method for image demosaicing provided by the embodiments of the present invention, polarization artifacts and jagged phenomena do not occur in complex texture regions, high-contrast regions, and non-oriented edge regions of the image, improving the quality of the image.

[0325] The above are only the preferred embodiments of the present invention and do not impose any limitation on the present invention. Any person skilled in the art within the technical field, without departing from the technical solution of the present invention, makes any form of equivalent substitution or modification and other changes to the technical solution and technical content disclosed by the present invention, which are all within the content of the technical solution of the present invention and still fall within the protection scope of the present invention.

Claims

1. An apparatus for image demosaicing, characterized in that, it includes: An image input unit for inputting a to-be-processed image in Bayer format, where the to-be-processed image includes red domain data, green domain data, and blue domain data; An image Log domain transformation unit for respectively mapping the red domain data, green domain data, and blue domain data to the Log domain, and performing gain processing on the red domain data, green domain data, and blue domain data after being mapped to the Log domain; A green domain initialization unit for initializing the green domain data after gain processing to obtain an initial value of the green domain; A red domain initialization unit for initializing the red domain data according to the initial value of the green domain to obtain an initial value of the red domain; A red-green similarity calculation unit for calculating the similarity between the initial value of the red domain and the initial value of the green domain to obtain a first similarity, and at the same time, for calculating the residual between the initial value of the red domain and the initial value of the green domain to obtain a first residual; A blue domain initialization unit for initializing the blue domain data according to the initial value of the green domain to obtain an initial value of the blue domain; A blue-green similarity calculation unit for calculating the similarity between the initial value of the blue domain and the initial value of the green domain to obtain a second similarity, and at the same time, for calculating the residual between the initial value of the blue domain and the initial value of the green domain to obtain a second residual; A green domain weighted calculation unit for calculating the final green domain data according to the initial value of the green domain, the first similarity, and the second similarity; A red-blue domain weighted calculation unit for calculating the final red domain data and the final blue domain data through the final green domain data, the first residual, and the second residual; An RGB image Log domain inverse transformation unit for transforming the final green domain data, the final blue domain data, and the final red domain data from the Log domain into an RGB image.

2. The apparatus for image demosaicing according to claim 1, characterized in that, it further includes: An RGB image output unit for outputting an RGB image.

3. The apparatus for image demosaicing according to claim 1, characterized in that, the red domain data includes R channel data, the green domain data includes Gr channel data and Gb channel data, and the blue domain data includes B channel data.

4. The apparatus for image demosaicing according to claim 3, characterized in that, the image Log domain transformation unit includes: A Log domain mapping module for respectively mapping the R channel data, Gr channel data, Gb channel data, and B channel data to the Log domain to obtain the R channel data, Gr channel data, Gb channel data, and B channel data after being mapped to the Log domain; A channel data gain compensation module for performing gain on the R channel data, Gr channel data, Gb channel data, and B channel data after being mapped to the Log domain.

5. The apparatus for image demosaicing according to claim 1, characterized in that, the green domain initialization unit includes: Green domain 0° direction initialization module, which is used to initialize the green domain data after gain processing in the 0° direction to obtain the initial value in the 0° direction of the green domain; Green domain 90° direction initialization module, which is used to initialize the green domain data after gain processing in the 90° direction to obtain the initial value in the 90° direction of the green domain.

6. The image demosaicing device according to claim 1, characterized in that, the red domain initialization unit includes: First 0° direction numerical extraction module, which is used to extract the initial value in the same row as the R channel in the 0° direction of the green domain and the value of the R channel itself; First two-dimensional wavelet filtering module, which is used to perform two-dimensional wavelet filtering on the initial value in the same row as the R channel in the 0° direction of the green domain to obtain the green feature component, green H component, green V component and green D component in the 0° direction, and is used to perform two-dimensional wavelet filtering on the value of the R channel itself to obtain the red feature component in the 0° direction; First red domain reference guided filtering module, which is used to perform guided filtering on the green feature component to obtain the entire distribution of the red domain feature component in the 0° direction; First two-dimensional wavelet synthesis module, which is used to perform inverse wavelet operation on the green H component, green V component and green D component in the 0° direction to obtain the red initial value in the 0° direction; First 90° direction numerical extraction module, which is used to extract the initial value in the same column as the R channel in the 90° direction of the green domain and the value of the R channel itself; Second two-dimensional wavelet filtering module, which is used to perform two-dimensional wavelet filtering on the initial value in the same column as the R channel in the 90° direction of the green domain to obtain the green feature component, green H component, green V component and green D component in the 90° direction, and is used to perform two-dimensional wavelet filtering on the value of the R channel itself to obtain the red feature component in the 90° direction; Second red domain reference guided filtering module, which is used to perform guided filtering on the green feature component in the 90° direction to obtain the entire distribution of the red domain feature component in the 90° direction; Second two-dimensional wavelet synthesis module, which is used to perform inverse wavelet operation on the green H component, green V component and green D component in the 90° direction to obtain the red initial value in the 90° direction.

7. The image demosaicing device according to claim 6, characterized in that, the red-green similarity calculation unit includes: First 0° direction similarity calculation module, which is used to calculate the similarity between the red domain and the green domain in the 0° direction; First 90° direction similarity calculation module, which is used to calculate the similarity between the red domain and the green domain in the 90° direction.

8. The image demosaicing device according to claim 1, characterized in that, the blue domain initialization unit includes: Second 0° direction numerical extraction module, which is used to extract the initial value in the same row as the B channel in the 0° direction of the green domain and the value of the B channel itself; The third two-dimensional wavelet filtering module is used to perform two-dimensional wavelet filtering on the initial values in the 0° direction of the green domain in the same row as the R channel to obtain the green feature component, green H component, green V component, and green D component in the 0° direction, and is also used to perform two-dimensional wavelet filtering on the original value of the B channel to obtain the blue feature component in the 0° direction; The first blue domain reference-guided filtering module is used to perform guided filtering on the blue feature component in the 0° direction to obtain the entire distribution of the blue domain feature component in the 0° direction; The third two-dimensional wavelet synthesis module is used to perform inverse wavelet operations on the green H component, green V component, and green D component in the 0° direction to obtain the blue initial value in the 0° direction; The second 90° direction value extraction module is used to extract the initial values in the 90° direction of the green domain in the same column as the B channel and the original value of the B channel; The fourth two-dimensional wavelet filtering module is used to perform two-dimensional wavelet filtering on the initial values in the 90° direction of the green domain in the same column as the B channel to obtain the green feature component, green H component, green V component, and green D component in the 90° direction, and is also used to perform two-dimensional wavelet filtering on the original value of the B channel to obtain the blue feature component in the 90° direction; The second blue domain reference-guided filtering module is used to perform guided filtering on the blue feature component in the 90° direction to obtain the entire distribution of the blue domain feature component in the 90° direction; The fourth two-dimensional wavelet synthesis module is used to perform guided filtering on the blue feature component in the 90° direction to obtain the entire distribution of the blue domain feature component in the 90° direction.

9. The image demosaicing device according to claim 1, characterized in that, the blue-green similarity calculation unit includes: The second 0° direction similarity calculation module is used to calculate the similarity between the blue domain and the green domain in the 0° direction; The second 90° direction similarity calculation module is used to calculate the similarity between the blue domain and the green domain in the 90° direction.

10. The image demosaicing device according to claim 1, characterized in that, the green domain weighted calculation unit includes: The weight calculation module is used to calculate the weights of the green domain in the 00°, 90°, 180°, and 270° directions respectively according to the green domain initial value, the similarity between the blue domain and the green domain, and the similarity between the red domain and the green domain; The weighted calculation module is used to calculate the final green domain data according to the weights of the green domain in the 00°, 90°, 180°, and 270° directions respectively.

11. An image demosaicing method using the image demosaicing device according to any one of claims 1 to 10, characterized in that, it includes: Input a Bayer format image to be processed, where the image to be processed includes red domain data, green domain data, and blue domain data; Map the red domain data, green domain data, and blue domain data to the Log domain respectively, and perform gain processing on the red domain data, green domain data, and blue domain data mapped to the Log domain; Initialize the green domain data after gain processing to obtain the green domain initial value; Initialize the red domain data according to the initial value of the green domain to obtain the initial value of the red domain; Calculate the similarity between the initial value of the red domain and the initial value of the green domain to obtain the first similarity. At the same time, calculate the residual between the initial value of the red domain and the initial value of the green domain to obtain the first residual; Initialize the blue domain data according to the initial value of the green domain to obtain the initial value of the blue domain; Calculate the similarity between the initial value of the blue domain and the initial value of the green domain to obtain the second similarity. At the same time, calculate the residual between the initial value of the blue domain and the initial value of the green domain to obtain the second residual; Calculate the final green domain data according to the initial value of the green domain, the first similarity and the second similarity; Calculate the final red domain data and the final blue domain data through the final green domain data, the first residual and the second residual; Transform the final green domain data, the final blue domain data and the final red domain data from the Log domain into an RGB image.

12. The method for demosaicing an image according to claim 11, wherein, after transforming the final green domain data, the final blue domain data and the final red domain data from the Log domain into an RGB image, it further includes: outputting the RGB image.

13. The method for demosaicing an image according to claim 11, wherein, the red domain data includes R channel data, the green domain data includes Gr channel data and Gb channel data, and the blue domain data includes B channel data.

14. The method for demosaicing an image according to claim 13, wherein, The method for mapping the red domain data, the green domain data and the blue domain data to the Log domain respectively and performing gain processing on the red domain data, the green domain data and the blue domain data mapped to the Log domain includes: Map the R channel data, Gr channel data, Gb channel data and B channel data to the Log domain respectively to obtain the R channel data, Gr channel data, Gb channel data and B channel data mapped to the Log domain; Perform gain on the R channel data, Gr channel data, Gb channel data and B channel data mapped to the Log domain.

15. The method for demosaicing an image according to claim 11, wherein, The method for initializing the green domain data after gain processing to obtain the initial value of the green domain includes: Perform initialization on the green domain data after gain processing in the 0° direction to obtain the initial value of the green domain in the 0° direction; Perform initialization on the green domain data after gain processing in the 90° direction to obtain the initial value of the green domain in the 90° direction.

16. The method for demosaicing an image according to claim 11, wherein, The method for initializing the red domain data according to the initial value of the green domain to obtain the initial value of the red domain includes: Extract the initial value in the same row as the R channel in the 0° direction of the green domain and the value of the R channel itself; Perform two-dimensional wavelet filtering on the initial value in the 0° direction of the green domain in the same row as the R channel to obtain the green feature component, green H component, green V component, and green D component in the 0° direction, and perform two-dimensional wavelet filtering on the original value of the R channel itself to obtain the red feature component in the 0° direction; Perform guided filtering on the green feature component to obtain the entire distribution of the red domain feature component in the 0° direction; Perform inverse wavelet operation on the green H component, green V component, and green D component in the 0° direction to obtain the red initial value in the 0° direction; Extract the initial value in the 90° direction of the green domain in the same column as the R channel and the original value of the R channel itself; Perform two-dimensional wavelet filtering on the initial value in the 90° direction of the green domain in the same column as the R channel to obtain the green feature component, green H component, green V component, and green D component in the 90° direction, and perform two-dimensional wavelet filtering on the original value of the R channel itself to obtain the red feature component in the 90° direction; Perform guided filtering on the green feature component in the 90° direction to obtain the entire distribution of the red domain feature component in the 90° direction; Perform inverse wavelet operation on the green H component, green V component, and green D component in the 90° direction to obtain the red initial value in the 90° direction.

17. The method for image demosaicing according to claim 16, characterized in that, The method for calculating the similarity between the red domain initial value and the green domain initial value to obtain the first similarity includes: Calculating the similarity between the red domain and the green domain in the 0° direction; Calculating the similarity between the red domain and the green domain in the 90° direction.

18. The method for image demosaicing according to claim 11, characterized in that, The method for initializing the blue domain data according to the green domain initial value to obtain the blue domain initial value includes: Extracting the initial value in the 0° direction of the green domain in the same row as the B channel and the original value of the B channel itself; Perform two-dimensional wavelet filtering on the initial value in the 0° direction of the green domain in the same row as the R channel to obtain the green feature component, green H component, green V component, and green D component in the 0° direction, and perform two-dimensional wavelet filtering on the original value of the B channel itself to obtain the blue feature component in the 0° direction; Perform guided filtering on the blue feature component in the 0° direction to obtain the entire distribution of the blue domain feature component in the 0° direction; Perform inverse wavelet operation on the green H component, green V component, and green D component in the 0° direction to obtain the blue initial value in the 0° direction; Extracting the initial value in the 90° direction of the green domain in the same column as the B channel and the original value of the B channel itself; Perform two-dimensional wavelet filtering on the initial value in the 90° direction of the green domain in the same column as the B channel to obtain the green feature component, green H component, green V component, and green D component in the 90° direction, and perform two-dimensional wavelet filtering on the original value of the B channel itself to obtain the blue feature component in the 90° direction; Perform guided filtering on the blue feature component in the 90° direction to obtain the entire distribution of the blue domain feature component in the 90° direction; Perform guided filtering on the blue feature component in the 90° direction to obtain the entire distribution of the blue domain feature component in the 90° direction.

19. The method for image demosaicing according to claim 11, characterized in that, The method for calculating the similarity between the initial value of the blue domain and the initial value of the green domain to obtain the second similarity includes: Calculate the similarity between the blue domain and the green domain in the 0° direction; Calculate the similarity between the blue domain and the green domain in the 90° direction.

20. The method for image demosaicing according to claim 11, characterized in that, The method for calculating the final green domain data according to the initial value of the green domain, the first similarity, and the second similarity includes: Calculate the weights of the green domain in the 00°, 90°, 180°, and 270° directions according to the initial value of the green domain, the similarity between the blue domain and the green domain, and the similarity between the red domain and the green domain; Calculate the final green domain data according to the weights of the green domain in the 00°, 90°, 180°, and 270° directions.

Images