G-channel demosaicing method and device based on RAW image in tetrabayer format
By calculating the weight factor and chromatic aberration estimates of pixel points of the four-Bayer format RAW image, the problem of difficulty in de-mosaicing processing of its G channel is solved, and the effective de-mosaic processing of the G channel of the four-Bayer format RAW image is realized, improving the image quality.
Patent Information
- Application Number
- CN202411880671.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-19
- Publication Date
- 2025-03-21
AI Technical Summary
It is difficult for the prior art to directly demosaic the G-channel of the four-Bayer format RAW images because its format is different from the ordinary Bayer format.
By obtaining the four-Bayer format RAW image, the weight factor and chromatic aberration estimate of the current pixel point are calculated, the G-channel value of the target pixel point is calculated based on these values, and the G-channel value of all target pixel points is counted to output the complete G-channel image.
Demosaic processing of the G-channel of the four Bayer format RAW images is realized, so that it can be processed using the existing demosaic processing method of ordinary Bayer format RAW images, and the imaging quality is improved.
Smart Images
Figure CN119676579A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a G channel de-mosaicing method and device based on a quad-Bayer format RAW image. Background Art
[0002] Image sensors are devices that convert optical images into electronic signals. They are widely used in digital cameras and other electronic optical devices. There is a huge difference in the sensitivity of image sensors in environments with sufficient light and insufficient light. For example, when there is insufficient light at night, the image sensor is prone to a decrease in signal-to-noise ratio, which makes the denoising effect in the later image signal processing process unsatisfactory.
[0003] In order to improve the imaging quality in dark environments, a new type of image sensor is proposed, whose RAW images are arranged in a quad-Bayer format. Since the arrangement of pixels in the RAW images in the quad-Bayer format is equivalent to splitting each pixel in the existing Bayer format into four pixels of the same color, the imaging quality can be significantly improved by merging effective pixels.
[0004] However, since the format of the Quad-Bayer RAW image is different from the existing ordinary Bayer format, the demosaicing algorithm in the existing mainstream image signal processing cannot be directly used for processing, and a demosaicing method suitable for the Quad-Bayer RAW image must be developed. In the demosaicing process, since the sampling rate of the G channel in the RAW image is higher than the sampling rate of the R channel and the B channel, the brightness and detail texture of the RGB image output after demosaicing are mainly determined by the G channel. Therefore, the most critical thing is to realize the demosaicing of the G channel of the Quad-Bayer RAW image. Summary of the invention
[0005] The object of the present invention is to provide a G channel demosaicing method and device based on a four-Bayer format RAW image, so as to solve the problem of how to perform demosaicing on the G channel of the four-Bayer format RAW image.
[0006] In order to solve the above technical problems, the present invention provides a G channel demosaicing method based on a four-Bayer format RAW image, comprising: Get quad-Bayer format RAW images; Calculate the weight factor of the current pixel; Calculate the color difference estimate of the current pixel; According to the weight factor and the color difference estimation value, the G channel value of the target pixel corresponding to the current pixel is calculated; Count the G channel values of all target pixels to output a complete G channel image.
[0007] Optionally, in the G channel demosaicing method based on the four-Bayer format RAW image, the method for calculating the weight factor of the current pixel includes: In the first buffer window, the color difference value of the current pixel is calculated; According to the color difference value, calculate the gradient information of the current pixel; According to the gradient information, the weight factor of the current pixel is calculated.
[0008] Optionally, in the G channel demosaicing method based on the four-Bayer format RAW image, the method for calculating the color difference value of the current pixel in the first cache window includes: Extracting a first cache window; If the current pixel is an R pixel or a B pixel, in the first cache window, the horizontal G channel estimation value of the current pixel is calculated using the pixels in the same row as the current pixel; the vertical G channel estimation value of the current pixel is calculated using the pixels in the same column as the current pixel; If the current pixel is a G pixel, then in the first cache window, the horizontal R channel estimation value or the horizontal B channel estimation value of the current pixel is calculated using the R pixel or the B pixel in the same row as the current pixel; the vertical R channel estimation value or the vertical B channel estimation value of the current pixel is calculated using the R pixel or the B pixel in the same column as the current pixel; The GR color difference value of the current pixel in the horizontal direction is calculated using the G channel estimation value and the R channel estimation value in the horizontal direction; the GB color difference value of the current pixel in the horizontal direction is calculated using the G channel estimation value and the B channel estimation value in the horizontal direction; the GR color difference value of the current pixel in the vertical direction is calculated using the G channel estimation value and the R channel estimation value in the vertical direction; the GB color difference value of the current pixel in the vertical direction is calculated using the G channel estimation value and the B channel estimation value in the vertical direction.
[0009] Optionally, in the G channel demosaicing method based on the four-Bayer format RAW image, the method for calculating the gradient information of the current pixel point according to the color difference value includes: Counting the GR color difference value in the horizontal direction and the GB color difference value in the horizontal direction to obtain a horizontal color difference image; counting the GR color difference value in the vertical direction and the GB color difference value in the vertical direction to obtain a vertical color difference image; In the horizontal color difference image, the horizontal gradient of the current pixel is calculated using the horizontal GR color difference value or the horizontal GB color difference value of the pixel in the same row as the current pixel; In the vertical color difference image, the vertical gradient of the current pixel is calculated by using the vertical GR color difference value and the vertical GB color difference value of the pixel in the same column as the current pixel.
[0010] Optionally, in the G channel demosaicing method based on the four-Bayer format RAW image, the method for calculating the gradient information of the current pixel point according to the color difference value further includes: Extracting a second cache window, where the current pixel is located in the middle row and the middle column; In the second cache window, a weighted average calculation is performed on the horizontal gradients of all pixels to obtain the final horizontal gradient of the current pixel; a weighted average calculation is performed on the vertical gradients of all pixels to obtain the final vertical gradient of the current pixel.
[0011] Optionally, in the G channel demosaicing method based on the quad-Bayer format RAW image, the method for calculating the weight factor of the current pixel point according to the gradient information includes: The inverse of the horizontal gradient of the current pixel plus 1 is used as the horizontal weight factor of the current pixel; the inverse of the vertical gradient of the current pixel plus 1 is used as the vertical weight factor of the current pixel; The sum of the horizontal weight factor and the vertical weight factor is used as the normalized weight of the current pixel.
[0012] Optionally, in the G channel demosaicing method based on the quad-Bayer format RAW image, the method for calculating the color difference estimation value of the current pixel includes: In the horizontal color difference image, the horizontal color difference estimation value of the current pixel is calculated by using the horizontal GR color difference value or the horizontal GB color difference value of the pixel in the same row as the current pixel; In the vertical color difference image, the vertical color difference estimation value of the current pixel is calculated by using the vertical GR color difference value and the vertical GB color difference value of the pixel in the same column as the current pixel.
[0013] Optionally, in the G channel demosaicing method based on the four-Bayer format RAW image, the method of calculating the G channel value of the target pixel point corresponding to the current pixel point according to the weight factor and the color difference estimation value includes: According to the weight factor and the color difference estimation value, the color difference value of the target pixel corresponding to the current pixel is calculated; The G channel value of the target pixel corresponding to the current pixel is calculated by using the color difference value of the target pixel corresponding to the current pixel and the original pixel value of the current pixel.
[0014] Optionally, in the G channel demosaicing method based on the quad-Bayer format RAW image, the method of calculating the color difference value of the target pixel corresponding to the current pixel according to the weight factor and the color difference estimation value includes: The horizontal direction weight factor and the vertical direction weight factor are used to perform weighted average calculation on the horizontal direction color difference estimation value and the vertical direction color difference estimation value of the current pixel point to obtain the color difference value of the target pixel point corresponding to the current pixel point.
[0015] In order to solve the above technical problems, the present invention also provides a G channel demosaicing device based on a four-Bayer format RAW image, comprising a memory, a processor, and an executable program stored in the memory and capable of being run by the processor; when the processor runs the executable program, the G channel demosaicing method based on a four-Bayer format RAW image as described in any one of the above items is executed.
[0016] The G channel demosaicing method and device based on a four-Bayer format RAW image provided by the present invention include: obtaining a four-Bayer format RAW image; calculating a weight factor of a current pixel; calculating a color difference estimation value of the current pixel; calculating the G channel value of a target pixel corresponding to the current pixel according to the weight factor and the color difference estimation value; and counting the G channel values of all target pixels to output a complete G channel image. By calculating the weight factor and the color difference estimation value for the pixel of the four-Bayer format RAW image, and calculating the G channel value of the target pixel of the ordinary Bayer format RAW image corresponding to the pixel of the four-Bayer format RAW image according to the weight factor and the color difference estimation value, the complete G channel value of the ordinary Bayer format RAW image corresponding to the four-Bayer format RAW image can be obtained, so that the G channel of the four-Bayer format RAW image can be demosaiced by using the existing demosaicing processing method for the ordinary Bayer format RAW image after the format conversion, and the problem of how to demosaic the G channel of the four-Bayer format RAW image is solved. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 A flowchart of a G channel demosaicing method based on a quad-Bayer format RAW image provided in this embodiment; Figure 2 A schematic diagram of pixel arrangement of a RAW image in a quad-Bayer format provided in this embodiment; FIG3 (a) is a schematic diagram of the structure of a horizontal G channel image provided in this embodiment; FIG3( b ) is a schematic structural diagram of a vertical G channel image provided in this embodiment; FIG3 (c) is a schematic diagram of the structure of a horizontal RB channel image provided in this embodiment; FIG3( d ) is a schematic structural diagram of a vertical RB channel image provided in this embodiment; FIG3 (e) is a schematic diagram of the structure of a horizontal color difference image provided by this embodiment; FIG3( f ) is a schematic structural diagram of a vertical chromatic aberration image provided by this embodiment; FIG3( g) is a schematic diagram of the structure of a horizontal gradient image provided by this embodiment; FIG3( h ) is a schematic structural diagram of a vertical gradient image provided by this embodiment. DETAILED DESCRIPTION
[0018] The G channel de-mosaicing method and device based on the four-Bayer format RAW image proposed by the present invention are further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the accompanying drawings are all in a very simplified form and use non-precise proportions, which are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention. In addition, the structure shown in the accompanying drawings is often a part of the actual structure. In particular, the emphasis that each accompanying drawing needs to show is different, and sometimes different proportions are used.
[0019] It should be noted that the terms "first", "second", etc. in the specification, claims and drawings of the present invention are used to distinguish similar objects in order to describe the embodiments of the present invention, rather than to describe a specific order or sequence. It should be understood that the structures used in this way can be interchanged under appropriate circumstances. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0020] This embodiment provides a G channel demosaicing method based on a four-Bayer format RAW image. Figure 1 As shown, including: S1, acquiring a four-Bayer format RAW image; S2, calculate the weight factor of the current pixel; S3, calculating the color difference estimation value of the current pixel; S4, calculating the G channel value of the target pixel corresponding to the current pixel according to the weight factor and the color difference estimation value; S5, counting the G channel values of all target pixels to output a complete G channel image.
[0021] The G channel demosaicing method based on the four-Bayer format RAW image provided in this embodiment calculates the weight factor and the color difference estimation value for the pixel point of the four-Bayer format RAW image, and obtains the G channel value of the pixel point of the four-Bayer format RAW image corresponding to the target pixel point of the ordinary Bayer format RAW image according to the weight factor and the color difference estimation value, so as to obtain the complete G channel value of the ordinary Bayer format RAW image corresponding to the four-Bayer format RAW image, so that after the G channel of the four-Bayer format RAW image is converted into a format, it can be demosaiced by using the existing demosaicing processing method for the ordinary Bayer format RAW image, thereby solving the problem of how to demosaic the G channel of the four-Bayer format RAW image.
[0022] Specifically, in this embodiment, in step S1, the method for obtaining a RAW image in a four-Bayer format may be to obtain an original RAW image in a four-Bayer format from various photographic and video equipment using a new type of image sensor.
[0023] Raw images are original image files obtained from image sensors of devices such as digital cameras, scanners, or film scanners. Raw images are not processed and therefore have internal colors with a wide color gamut, allowing for precise adjustments.
[0024] In a RAW image of the four-Bayer format, two adjacent G pixels in each row are separated by two adjacent R pixels or two B pixels, and there are only G pixels and R pixels or G pixels and B pixels in the same row; the pixel arrangement of every two rows is consistent, and the G pixels in the first two rows and the last two rows are staggered, and the G pixels in the first two rows and the last two rows are separated by R pixels and B pixels respectively; the arrangement of columns is consistent with that of rows. For example, Figure 2 As shown, when two adjacent G pixels in the i-th row are separated by two adjacent R pixels, the two adjacent G pixels in the i+1-th row are also separated by two adjacent R pixels, and the pixel arrangement of the i+1-th row is consistent with the pixel arrangement of the i-th row; the two adjacent G pixels in the i+2-th row are separated by two adjacent B pixels, and the two G pixels are staggered with the two G pixels in the i-th row; the two adjacent G pixels in the i+3-th row are also separated by two adjacent B pixels, and the pixel arrangement of the i+3-th row is consistent with the pixel arrangement of the i+2-th row.
[0025] Furthermore, in this embodiment, in step S2, the method for calculating the weight factor of the current pixel includes: S21, calculating the color difference value of the current pixel in the first cache window.
[0026] Specifically, in this embodiment, first, the first cache window is extracted. The size of the first cache window is M×N, which should contain a sufficient number of pixels for calculating the color difference value of the current pixel. In the first cache window, (i, j) indicates that the pixel is located at the i+1th row and j+1th column in the first cache window, and D(i, j) represents the pixel value of the pixel at the position (i, j) in the first cache window, i=0,1,2…M-1, j=0,1,2…N-1. In this embodiment, a 10×10 first cache window can be selected to calculate the color difference value of the current pixel.
[0027] Then, considering that in the four-Bayer format RAW image, the arrangement rules of R pixels and B pixels are consistent, while the arrangement rules of G pixels are inconsistent with those of R pixels and B pixels, it is necessary to calculate R pixels, B pixels and G pixels separately, where: If the current pixel is an R pixel or a B pixel, in the first cache window, the horizontal G channel estimation value of the current pixel is calculated using the pixels in the same row as the current pixel; the vertical G channel estimation value of the current pixel is calculated using the pixels in the same column as the current pixel.
[0028] In this embodiment, a horizontal G channel estimation value is provided when the current pixel is an R pixel or a B pixel. The calculation method is expressed as:
[0029] And, provide a vertical G channel estimation value when the current pixel is an R pixel or a B pixel The calculation method is expressed as:
[0030] In a specific embodiment, Figure 2 Take the pixel structure shown as an example, where the coordinate (4,4) in the first cache window is an R pixel (a four-Bayer pixel at the center of the first cache window), so it is necessary to calculate the horizontal G channel estimation value and the vertical G channel estimation value for it. Using the above formula, the horizontal G channel estimation value and the vertical G channel estimation value of the R pixel at the coordinate (4,4) are respectively:
[0031]
[0032] Similarly, the horizontal G channel estimation value and the vertical G channel estimation value of the R pixel at other coordinate positions can be calculated. Also, the calculation method of the B pixel is consistent with the calculation method of the R pixel, and this application will not repeat it.
[0033] The calculated horizontal G channel estimation value of the R pixel and the horizontal G channel estimation value of the B pixel are combined with the G channel value of the original G pixel to obtain a horizontal G channel image, as shown in Figure 3 (a); similarly, the calculated vertical G channel estimation value of the R pixel and the vertical G channel estimation value of the B pixel are combined with the G channel value of the original G pixel to obtain a vertical G channel image, as shown in Figure 3 (b).
[0034] If the current pixel is a G pixel, then in the first cache window, the horizontal R channel estimation value or the horizontal B channel estimation value of the current pixel is calculated using the R pixel or B pixel in the same row as the current pixel; the vertical R channel estimation value or the vertical B channel estimation value of the current pixel is calculated using the R pixel or B pixel in the same column as the current pixel.
[0035] In this embodiment, a horizontal R channel estimation value is provided when the current pixel is a G pixel. or horizontal B channel estimate The calculation method is expressed as:
[0036]
[0037] And, provide a vertical R channel estimation value when the current pixel is a G pixel Or vertical B channel estimate , expressed as:
[0038]
[0039] In a specific embodiment, Figure 2 Take the pixel structure shown as an example, where the coordinate (4,2) in the first cache window is a G pixel, and the same row is an R pixel, so it is necessary to calculate the horizontal R channel estimation value for it. Using the above formula, the horizontal R channel estimation value of the G pixel at the coordinate (4,2) is:
[0040] And, the coordinate (4,2) is a G pixel, and the pixel in the same column is a B pixel. Since it is necessary to calculate the vertical B channel estimation value, the vertical B channel estimation value of the G pixel at the coordinate (4,2) can be obtained by using the above formula:
[0041] Similarly, the horizontal R channel estimation value or the horizontal B channel estimation value and the vertical R channel estimation value or the vertical B channel estimation value of the G pixel at other coordinate positions can be calculated. It should be noted that for the same G pixel, when the same row is an R pixel, only the horizontal R channel estimation value is calculated, and the pixels in the same column must be B pixels, and only the vertical B channel estimation value is calculated; similarly, when the same row is a B pixel, only the horizontal B channel estimation value is calculated, and the pixels in the same column must be R pixels, and only the vertical R channel estimation value is calculated.
[0042] The calculated horizontal R channel estimation value and horizontal B channel estimation value of the G pixel are combined with the R channel value of the original R pixel and the B channel value of the original B pixel to obtain a horizontal RB channel image, as shown in Figure 3 (c); similarly, the calculated vertical R channel estimation value and vertical B channel estimation value of the G pixel are combined with the R channel value of the original R pixel and the B channel value of the original B pixel to obtain a vertical RB channel image, as shown in Figure 3 (d).
[0043] Finally, the GR color difference value in the horizontal direction of the current pixel is calculated using the estimated value of the G channel and the estimated value of the R channel in the horizontal direction, that is, the GR color difference value in the horizontal direction is obtained by using Gh-Rh, recorded as GRh; the GB color difference value in the horizontal direction of the current pixel is calculated using the estimated value of the G channel and the B channel in the horizontal direction, that is, the GB color difference value in the horizontal direction is obtained by using Gh-Bh, recorded as GBh.
[0044] The horizontal GR color difference value GRh and the horizontal GB color difference value GBh are counted to obtain a horizontal color difference image, as shown in FIG3( e).
[0045] And, using the G channel estimation value and the R channel estimation value in the vertical direction, the GR color difference value in the vertical direction of the current pixel is calculated, that is, the GR color difference value in the vertical direction is obtained by using Gv-Rv, recorded as GRv; using the G channel estimation value and the B channel estimation value in the vertical direction, the GB color difference value in the vertical direction of the current pixel is calculated, that is, the GB color difference value in the vertical direction is obtained by using Gv-Bv, recorded as GBv.
[0046] The GR color difference value GRv in the vertical direction and the GB color difference value GBv in the vertical direction are counted to obtain a vertical color difference image, as shown in FIG3( f ).
[0047] It can be seen from the horizontal color difference image (Figure 3 (e)) and the vertical color difference image (Figure 3 (f)) that in the horizontal color difference image, the color difference type of pixels in the same row is consistent, which is GB color difference or GR color difference, and the color difference types of every two rows of pixels are staggered; in the vertical color difference image, the color difference type of pixels in the same column is consistent, which is GB color difference or GR color difference, and the color difference types of every two columns of pixels are staggered. This rule is also in line with the pixel distribution rule of the four-Bayer format RAW image.
[0048] Of course, in other embodiments, the channel estimation value may also be calculated in other ways, such as selecting a first cache window of other sizes, selecting pixels at other intervals for addition and subtraction operations, or setting different weights for different pixel differences, etc. This application does not impose any restrictions on this.
[0049] S22, calculating the gradient information of the current pixel according to the color difference value.
[0050] Specifically, in this embodiment, in the horizontal color difference image, the horizontal gradient of the current pixel can be calculated by using the horizontal GR color difference value or the horizontal GB color difference value of the pixels in the same row as the current pixel; and in the vertical color difference image, the vertical gradient of the current pixel can be calculated by using the vertical GR color difference value and the vertical GB color difference value of the pixels in the same column as the current pixel.
[0051] In this embodiment, a method is provided for calculating the horizontal gradient of the current pixel point by using the horizontal GR color difference value or the horizontal GB color difference value of the pixel point in the same row as the current pixel point in the horizontal color difference image. The calculation method is expressed as:
[0052] And, a method for calculating the vertical gradient of the current pixel point by using the vertical GR color difference value and the vertical GB color difference value of the pixel point in the same column as the current pixel point in the vertical color difference image is provided. , expressed as:
[0053] in,( ) indicates that the pixel is located at the u+1th row and the v+1th column in the horizontal color difference image or the vertical color difference image. Indicated in the horizontal color difference image ( ) horizontal color difference value of the pixel at the position, including GRh and GBh; Indicated in the vertical chromatic aberration image ( ) vertical color difference value of the pixel at the position, including GRv and GBv; u = 0, 1, 2 ... U-1, v = 0, 1, 2 ... V-1, the size of the horizontal color difference image or the vertical color difference image is U × V. In practical applications, the size of the horizontal color difference image or the vertical color difference image should be consistent with the size of the original four-Bayer format RAW image.
[0054] In a specific embodiment, taking the pixel structure shown in FIG. 3 (e) as an example, the horizontal gradient of the pixel point at the coordinate (2,2) is It can be expressed as:
[0055] And, taking the pixel structure shown in FIG3(f) as an example, the vertical gradient of the pixel at the coordinate (2,2) can be expressed as:
[0056] Similarly, the horizontal gradient and vertical gradient of pixel points at other coordinate positions can be calculated.
[0057] The horizontal gradient of each pixel is counted to obtain a horizontal gradient image, as shown in FIG3(g); the vertical gradient of each pixel is counted to obtain a vertical gradient image, as shown in FIG3(h).
[0058] Of course, in other embodiments, the calculation of the horizontal gradient and the vertical gradient may also be performed using other calculation methods. For example, when calculating the horizontal gradient, the intervals between similar pixels in the same row may have other settings, and the present application does not impose any restrictions on this.
[0059] Preferably, in order to further improve the calculation accuracy of the gradient and ensure that the converted G channel value can retain the image detail information, in this embodiment, step S22, the method for calculating the gradient information of the current pixel point according to the color difference value also includes: First, extract the second cache window, in which the current pixel is located in the middle row and middle column. For ease of calculation, the second cache window extracted in the horizontal gradient image and the vertical gradient image is of the same size, for example, a second cache window of size 5×5 is extracted. Of course, in practical applications, a rectangular window can also be extracted as the second cache window according to the gradient direction. For example, for a horizontal gradient image, a 3×5 second cache window is extracted; for a numerical gradient image, a 5×3 second cache window is extracted. This application does not limit the size of the second cache window.
[0060] Then, in the second cache window, a weighted average calculation is performed on the horizontal gradients of all pixels to obtain the final horizontal gradient of the current pixel; and a weighted average calculation is performed on the vertical gradients of all pixels to obtain the final vertical gradient of the current pixel.
[0061] In a specific embodiment, taking the pixel structure shown in FIG. 3( g) as an example, where the size of the second cache window is 3×5, the final horizontal gradient of the pixel at the coordinate (2,2) can be expressed as:
[0062] And, taking the pixel structure shown in FIG3(h) as an example, where the size of the second cache window is 5×3, the final vertical gradient of the pixel at the coordinate (2,2) can be expressed as:
[0063] Of course, in other embodiments, when the size of the second cache window changes, the corresponding final gradient calculation method also changes. Also, the weights set for the gradient values of each pixel point may have other values, which are not limited in this application.
[0064] S23, calculating the weight factor of the current pixel according to the gradient information.
[0065] Specifically, in this embodiment, the inverse of the horizontal gradient of the current pixel plus 1 is used as the horizontal weight factor of the current pixel. ,Right now:
[0066] The inverse of the vertical gradient of the current pixel plus 1 is used as the vertical weight factor of the current pixel ,Right now:
[0067] And, the sum of the horizontal weight factor and the vertical weight factor is used as the normalized weight of the current pixel ,Right now:
[0068] Furthermore, in this embodiment, in step S3, the method for calculating the color difference estimation value of the current pixel point includes: In the horizontal color difference image, the horizontal color difference estimation value of the current pixel is calculated using the horizontal GR color difference value or the horizontal GB color difference value of the pixels in the same row as the current pixel; and in the vertical color difference image, the vertical color difference estimation value of the current pixel is calculated using the vertical GR color difference value and the vertical GB color difference value of the pixels in the same column as the current pixel.
[0069] This embodiment provides a horizontal color difference estimation value of the current pixel point. The calculation method is expressed as:
[0070] And, give an estimate of the vertical color difference of the current pixel The calculation method is expressed as:
[0071] In a specific embodiment, taking the pixel structure shown in FIG. 3 (e) as an example, the estimated value of the horizontal color difference of the pixel at the coordinate (2,2) is expressed as:
[0072] And, taking the pixel structure shown in FIG3(f) as an example, the vertical color difference estimation value of the pixel at the coordinate (2,2) is expressed as:
[0073] Similarly, the estimated values of the horizontal color difference and the vertical color difference of the pixel points at other coordinate positions can be calculated.
[0074] Of course, in other embodiments, the color difference estimation value may be calculated in other ways, such as changing the weights of each color difference value in the calculation formula, or introducing a greater or lesser number of pixels to participate in the calculation, etc., and the present application does not impose any limitation on this.
[0075] Further, in this embodiment, in step S4, the method of calculating the G channel value of the target pixel corresponding to the current pixel according to the weight factor and the color difference estimation value includes: S41, calculating the color difference value of the target pixel point corresponding to the current pixel point according to the weight factor and the color difference estimation value.
[0076] Specifically, in this embodiment, the horizontal direction weight factor and the vertical direction weight factor are used to perform weighted average calculation on the horizontal direction color difference estimation value and the vertical direction color difference estimation value of the current pixel point to obtain the color difference value of the target pixel point corresponding to the current pixel point.
[0077] Taking the pixel at coordinate (2,2) as an example, its horizontal weight factor is expressed as Wh(2,2), and the estimated horizontal color difference is , the vertical weight factor is expressed as Wv(2,2), and the vertical color difference estimation value is , the normalized weight is expressed as Wt(2,2), which is the R pixel in the original RAW image of the four-Bayer format. Then, the color difference value of the target pixel corresponding to the R pixel at the coordinate (2,2) can be expressed as:
[0078] Similarly, the color difference value of the target pixel corresponding to the R pixel or B pixel at other coordinate positions can be calculated.
[0079] S42, using the color difference value of the target pixel corresponding to the current pixel and the original pixel value of the current pixel, calculate the G channel value of the target pixel corresponding to the current pixel.
[0080] Specifically, in this embodiment, the G channel value of the target pixel corresponding to the current pixel is obtained by summing the color difference value calculated in step S41 and the original pixel value of the current pixel.
[0081] Still taking the pixel at coordinate (2,2) as an example, the G channel value of the corresponding target pixel can be expressed as:
[0082] In this way, all R pixels and B pixels in the original RAW image in the four-Bayer format are calculated to obtain the G channel value corresponding to each pixel.
[0083] Furthermore, in this embodiment, in step S5, the G channel values of all target pixels are counted to output a complete G channel image.
[0084] It should be noted that the four pixels located in the center of Figure 3 (a) to Figure 3 (h) correspond to Figure 2 The four pixels at the center, i.e., the pixels at coordinates (2,2) in Figures 3(a) to 3(h), correspond to Figure 2 The pixel at coordinate (4,4).
[0085] The G channel demosaicing method based on the four-Bayer format RAW image provided in this embodiment calculates the weight factor and the color difference estimation value for the pixel point of the four-Bayer format RAW image, and obtains the G channel value of the pixel point of the four-Bayer format RAW image corresponding to the target pixel point of the ordinary Bayer format RAW image according to the weight factor and the color difference estimation value, so as to obtain the complete G channel value of the ordinary Bayer format RAW image corresponding to the four-Bayer format RAW image, so that after the G channel of the four-Bayer format RAW image is converted into a format, it can be demosaiced by using the existing demosaicing processing method for the ordinary Bayer format RAW image, thereby solving the problem of how to demosaic the G channel of the four-Bayer format RAW image.
[0086] The G channel demosaicing method based on the four-Bayer format RAW image provided in this embodiment has a simple calculation method for the weight factor and the color difference estimation value, involves a small amount of data, occupies less hardware resources, and has a high calculation efficiency; at the same time, the G channel demosaicing method based on the four-Bayer format RAW image provided in this embodiment has a high accuracy of the calculated G channel value of the target pixel point corresponding to the current pixel point, so that the texture details of the image can be accurately retained after the demosaicing process, thereby improving the image quality.
[0087] This embodiment also provides a G channel demosaicing device based on a four-Bayer format RAW image, comprising a memory, a processor, and an executable program stored in the memory and capable of being run by the processor; when the processor runs the executable program, the G channel demosaicing method based on the four-Bayer format RAW image as described above is executed.
[0088] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other. In addition, the different parts between the various embodiments can also be used in combination with each other, and the present invention is not limited to this.
[0089] The G channel demosaicing method and device based on the four-Bayer format RAW image provided in this embodiment include: obtaining the four-Bayer format RAW image; calculating the weight factor of the current pixel; calculating the color difference estimation value of the current pixel; calculating the G channel value of the target pixel corresponding to the current pixel according to the weight factor and the color difference estimation value; and counting the G channel values of all the target pixels to output a complete G channel image. By calculating the weight factor and the color difference estimation value for the pixel of the four-Bayer format RAW image, and calculating the G channel value of the target pixel of the ordinary Bayer format RAW image corresponding to the pixel of the four-Bayer format RAW image according to the weight factor and the color difference estimation value, the complete G channel value of the ordinary Bayer format RAW image corresponding to the four-Bayer format RAW image can be obtained, so that the G channel of the four-Bayer format RAW image can be demosaiced by using the existing demosaicing processing method for the ordinary Bayer format RAW image after the format conversion, and the problem of how to demosaic the G channel of the four-Bayer format RAW image is solved.
[0090] The above description is only a description of the preferred embodiments of the present invention, and is not intended to limit the scope of the present invention. Any changes or modifications made by a person skilled in the art in the field of the present invention based on the above disclosure shall fall within the scope of protection of the claims.
Claims
1. A G channel demosaicing method based on a quad-Bayer format RAW image, characterized in that: include: Get quad-Bayer format RAW images; Calculate the weight factor of the current pixel; Calculate the color difference estimate of the current pixel; According to the weight factor and the color difference estimation value, the G channel value of the target pixel corresponding to the current pixel is calculated; Count the G channel values of all target pixels to output a complete G channel image.
2. The G channel demosaicing method based on a quad-Bayer format RAW image according to claim 1, characterized in that: The method for calculating the weight factor of the current pixel point includes: In the first buffer window, the color difference value of the current pixel is calculated; According to the color difference value, calculate the gradient information of the current pixel; According to the gradient information, the weight factor of the current pixel is calculated.
3. The G channel demosaicing method based on a quad-Bayer format RAW image according to claim 2, characterized in that: The method for calculating the color difference value of the current pixel in the first cache window includes: Extracting a first cache window; If the current pixel is an R pixel or a B pixel, in the first cache window, the horizontal G channel estimation value of the current pixel is calculated using the pixels in the same row as the current pixel; the vertical G channel estimation value of the current pixel is calculated using the pixels in the same column as the current pixel; If the current pixel is a G pixel, then in the first cache window, the horizontal R channel estimation value or the horizontal B channel estimation value of the current pixel is calculated using the R pixel or the B pixel in the same row as the current pixel; the vertical R channel estimation value or the vertical B channel estimation value of the current pixel is calculated using the R pixel or the B pixel in the same column as the current pixel; The GR color difference value of the current pixel in the horizontal direction is calculated using the G channel estimation value and the R channel estimation value in the horizontal direction; the GB color difference value of the current pixel in the horizontal direction is calculated using the G channel estimation value and the B channel estimation value in the horizontal direction; the GR color difference value of the current pixel in the vertical direction is calculated using the G channel estimation value and the R channel estimation value in the vertical direction; the GB color difference value of the current pixel in the vertical direction is calculated using the G channel estimation value and the B channel estimation value in the vertical direction.
4. The G channel demosaicing method based on a quad-Bayer format RAW image according to claim 3, characterized in that: The method for calculating the gradient information of the current pixel point according to the color difference value includes: Counting the GR color difference value in the horizontal direction and the GB color difference value in the horizontal direction to obtain a horizontal color difference image; counting the GR color difference value in the vertical direction and the GB color difference value in the vertical direction to obtain a vertical color difference image; In the horizontal color difference image, the horizontal gradient of the current pixel is calculated using the horizontal GR color difference value or the horizontal GB color difference value of the pixel in the same row as the current pixel; In the vertical color difference image, the vertical gradient of the current pixel is calculated by using the vertical GR color difference value and the vertical GB color difference value of the pixel in the same column as the current pixel.
5. The G channel demosaicing method based on a quad-Bayer format RAW image according to claim 4, characterized in that: The method for calculating the gradient information of the current pixel point according to the color difference value also includes: Extracting a second cache window, where the current pixel is located in the middle row and the middle column; In the second cache window, a weighted average calculation is performed on the horizontal gradients of all pixels to obtain the final horizontal gradient of the current pixel; a weighted average calculation is performed on the vertical gradients of all pixels to obtain the final vertical gradient of the current pixel.
6. The G channel demosaicing method based on a quad-Bayer format RAW image according to claim 4 or 5, characterized in that: The method for calculating the weight factor of the current pixel point according to the gradient information includes: The inverse of the horizontal gradient of the current pixel plus 1 is used as the horizontal weight factor of the current pixel; the inverse of the vertical gradient of the current pixel plus 1 is used as the vertical weight factor of the current pixel; The sum of the horizontal weight factor and the vertical weight factor is used as the normalized weight of the current pixel.
7. The G channel demosaicing method based on a quad-Bayer format RAW image according to claim 6, characterized in that: The method for calculating the color difference estimation value of the current pixel point includes: In the horizontal color difference image, the horizontal color difference estimation value of the current pixel is calculated by using the horizontal GR color difference value or the horizontal GB color difference value of the pixel in the same row as the current pixel; In the vertical color difference image, the vertical color difference estimation value of the current pixel is calculated by using the vertical GR color difference value and the vertical GB color difference value of the pixel in the same column as the current pixel.
8. The G channel demosaicing method based on a quad-Bayer format RAW image according to claim 7, characterized in that: The method for calculating the G channel value of the target pixel point corresponding to the current pixel point according to the weight factor and the color difference estimation value includes: According to the weight factor and the color difference estimation value, the color difference value of the target pixel corresponding to the current pixel is calculated; The G channel value of the target pixel corresponding to the current pixel is calculated by using the color difference value of the target pixel corresponding to the current pixel and the original pixel value of the current pixel.
9. The G channel demosaicing method based on a quad-Bayer format RAW image according to claim 8, characterized in that: The method for calculating the color difference value of the target pixel point corresponding to the current pixel point according to the weight factor and the color difference estimation value includes: The horizontal direction weight factor and the vertical direction weight factor are used to perform weighted average calculation on the horizontal direction color difference estimation value and the vertical direction color difference estimation value of the current pixel point to obtain the color difference value of the target pixel point corresponding to the current pixel point.
10. A G channel demosaicing device based on a quad-Bayer format RAW image, characterized in that: The invention comprises a memory, a processor and an executable program stored in the memory and capable of being run by the processor; when the processor runs the executable program, the G channel demosaicing method based on a four-Bayer format RAW image as described in any one of claims 1 to 9 is executed.