Image demosaicing method and device based on RGBW pixels, electronic equipment and computer storage medium

Abstract

The invention provides an image demosaicing method and device based on RGBW pixels, electronic equipment and a computer storage medium, and the method comprises the steps: obtaining a full-resolution W brightness image through W pixel information in an original RAW image; calculating gradient information of an RGB channel by using the W pixel information and the W brightness image, and further calculating a W pixel value of each pixel in the original RAW image to obtain a W interpolation image; and performing interpolation and enhancement processing on the RGB channel by using the W interpolation image to obtain an RGB image. Gradient information of an RGB channel is calculated by introducing a W brightness image, and then interpolation is performed on pixels, so that image detail textures can be effectively reserved; meanwhile, the RGB channel is enhanced through the W interpolation image, image details and textures can be better restored, moire patterns, aliasing and the like are reduced, the quality of the RGB image obtained through demosaicing is effectively improved, and the problem of how to carry out demosaicing processing on the image of the RGBW pixel structure is solved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image de-mosaic method and apparatus, electronic device and computer storage medium based on RGBW pixels. Background Technology

[0002] With the development of display technology, the RGBW pixel structure has emerged based on the RGB pixel structure. The RGBW pixel structure includes pixels of four colors: white, red, green, and blue. By adding white pixels, the image has higher brightness, richer colors, and lower power consumption, making it widely used in 4K resolution display scenarios.

[0003] To restore an RGB image from a raw RAW image obtained using an RGBW pixel structure, demosaicing is required. Existing demosaicing methods are all based on the RGB pixel format, that is, interpolating the G pixel (with the highest sampling rate) using R and B pixels. However, the photosensitive characteristics of the W pixel in the RGBW pixel structure are different from those of the RGB pixels. Therefore, the RGB channels cannot be directly used to assist in interpolating the W pixel. In other words, existing image demosaicing methods based on RGB pixels cannot be applied to RGBW pixel images. Summary of the Invention

[0004] The purpose of this invention is to provide an image demosaic method, apparatus, electronic device, and computer storage medium based on RGBW pixels, so as to at least solve the problem of how to perform demosaic processing on images with RGBW pixel structure.

[0005] To address the aforementioned technical problems, this invention provides an image demosaic method based on RGBW pixels, comprising: Obtain the original RAW image with RGBW pixel structure; By utilizing the W pixel information in the original RAW image, a full-resolution W brightness image is obtained; Calculate the gradient information of the RGB channels using the W pixel information and W brightness image in the original RAW image; Using the gradient information of the RGB channels, the W pixel value of each pixel in the original RAW image is calculated to obtain the W interpolated image. The W interpolation image is used to interpolate and enhance the RGB channels to obtain an RGB image.

[0006] Optionally, in the image demosaic method based on RGBW pixels, the method for obtaining a full-resolution W luminance image using the W pixel information in the original RAW image includes: The W channel of the original RAW image is downsampled by 1 / 4 to obtain a downsampled image; The downsampled image is upsampled back to its original resolution to obtain the W brightness image.

[0007] Optionally, in the image demosaicing method based on RGBW pixels, the method for calculating the gradient information of the RGB channels using the W pixel information and W luminance image in the original RAW image includes: Using the pixel value of pixel W in the original RAW image, the gradient values ​​of the RGB channels in multiple directions and at multiple frequencies are calculated; the directions include the horizontal direction, the vertical direction, and the diagonal direction; the frequency represents the distance between the current pixel and the pixel used to calculate the gradient of the current pixel, wherein the greater the distance, the lower the frequency. Based on the W luminance image, the gradient values ​​of multiple directions and multiple frequencies of the RGB channels are fused to obtain the fused gradient values ​​of the RGB channels.

[0008] Optionally, in the image demosaic method based on RGBW pixels, the method for fusing gradient values ​​of multiple directions and multiple frequencies of the RGB channels based on the W luminance image to obtain the fused gradient values ​​of the RGB channels includes: Based on the W luminance image, the gradient values ​​of each direction and each frequency of the RGB channel are normalized to obtain normalized gradient values ​​of multiple directions and multiple frequencies. The normalized gradient values ​​of all frequencies in the same direction of the RGB channels are weighted and fused to obtain the fused gradient value in that direction.

[0009] Optionally, in the image demosaicing method based on RGBW pixels, the method for calculating the gradient information of the RGB channels using the W pixel information and W luminance image in the original RAW image further includes: The horizontal and vertical fusion gradient values ​​are used to correct the diagonal fusion gradient value.

[0010] Optionally, in the image demosaic method based on RGBW pixels, the method of calculating the W pixel value of each pixel in the original RAW image using the gradient information of the RGB channels to obtain the W interpolated image includes: Calculate the W interpolation weights using the gradient information of the RGB channels; Calculate the W pixel value of the RGB pixel using the W interpolation weight; Use the original pixel value of pixel W as the W pixel value of pixel W; Combine the W pixel values ​​of the RGB pixels and the W pixel values ​​of the W pixels to obtain the W interpolated image.

[0011] Optionally, in the image demosaic method based on RGBW pixels, the method of interpolating and enhancing the RGB channels using W interpolation to obtain an RGB image includes: Calculate the energy intensity gain of the original RAW image; The W interpolated image is multiplied by the energy intensity gain to obtain the W enhanced image; The RGB channels in the W-enhanced image and the original RAW image are downsampled by 1 / 16 to obtain the W-downsampled image, R-downsampled image, G-downsampled image and B-downsampled image; The W-downsampled image is subtracted from the R-downsampled image, G-downsampled image, and B-downsampled image respectively to obtain the WR-downsampled difference image, WG-downsampled difference image, and WB-downsampled difference image. Upsample the WR downsampled difference image, WG downsampled difference image, and WB downsampled difference image back to their original resolution to obtain the WR difference image, WG difference image, and WB difference image; The W-enhanced image is subtracted from the WR difference image, WG difference image, and WB difference image respectively to obtain the first R image, the first G image, and the first B image; Combine the first R image, the first G image, and the first B image to obtain an RGB image.

[0012] Optionally, in the image demosaic method based on RGBW pixels, after obtaining the first R image, the first G image, and the first B image, the method of interpolating and enhancing the RGB channels using the W interpolation image to obtain the RGB image further includes: The R pixel values ​​in the first R image are updated to obtain the second R image, wherein: for an R pixel in the original RAW image, the difference between its window filter value in the first R image and its R pixel value in the original RAW image is calculated; if the difference is greater than a preset R difference threshold, the R pixel value in the first R image is used as the R pixel value of that R pixel; otherwise, the R pixel values ​​in the first R image and the R pixel values ​​in the original RAW image are weighted and fused to obtain the updated R pixel value of that R pixel. The G-pixel values ​​in the first G-image are updated to obtain the second G-image, wherein: for a G-pixel in the original RAW image, the difference between its window filter value in the first G-image and its G-pixel value in the original RAW image is calculated; if the difference is greater than a preset G difference threshold, the G-pixel value in the first G-image is used as the G-pixel value of that G-pixel; otherwise, the G-pixel values ​​in the first G-image and the G-pixel values ​​in the original RAW image are weighted and fused to obtain the updated G-pixel value of that G-pixel. The B pixel values ​​in the first B image are updated to obtain the second B image, wherein: for a B pixel in the original RAW image, the difference between its window filter value in the first B image and its B pixel value in the original RAW image is calculated; if the difference is greater than a preset B difference threshold, the B pixel value in the first B image is used as the B pixel value of that B pixel; otherwise, the B pixel values ​​in the first B image and the B pixel values ​​in the original RAW image are weighted and fused to obtain the updated B pixel value of that B pixel. Combine the second R image, the second G image, and the second B image to obtain an RGB image.

[0013] To address the aforementioned technical problems, the present invention also provides an image de-mosaic device based on RGBW pixels, used to implement the image de-mosaic method based on RGBW pixels as described in any of the preceding claims, wherein the image de-mosaic device based on RGBW pixels comprises: The image acquisition module is used to acquire the original RAW image with RGBW pixel structure; The brightness calculation module is used to obtain a full-resolution W brightness image by utilizing the W pixel information in the original RAW image; The gradient calculation module is used to calculate the gradient information of the RGB channels using the W pixel information and W brightness image in the original RAW image. The interpolation processing module is used to calculate the W pixel value of each pixel in the original RAW image using the gradient information of the RGB channels, so as to obtain the W interpolated image. The enhancement processing module is used to interpolate and enhance the RGB channels using the W interpolated image to obtain an RGB image.

[0014] To address the aforementioned technical problems, the present invention also provides an electronic device, including a memory, a processor, and an executable program stored in the memory and capable of being run by the processor; when the processor runs the executable program, it executes the image de-mosaic method based on RGBW pixels as described in any of the preceding claims.

[0015] To address the aforementioned technical problems, the present invention also provides a computer storage medium storing an executable program; when the executable program is executed, it implements the image de-mosaic method based on RGBW pixels as described in any of the preceding claims.

[0016] This invention provides an image de-mosaic method, apparatus, electronic device, and computer storage medium based on RGBW pixels, comprising: acquiring an original RAW image with an RGBW pixel structure; obtaining a full-resolution W luminance image using the W pixel information in the original RAW image; calculating the gradient information of the RGB channels using the W pixel information and the W luminance image in the original RAW image; calculating the W pixel value of each pixel in the original RAW image using the gradient information of the RGB channels to obtain a W interpolated image; and performing interpolation and enhancement processing on the RGB channels using the W interpolated image to obtain an RGB image. By introducing the W luminance image to calculate the gradient information of the RGB channels and then interpolating the pixels, the image details and textures can be effectively preserved. Simultaneously, by enhancing the RGB channels using the W interpolated image, image details and textures can be better restored, and moiré patterns and aliasing can be reduced, effectively improving the quality of the RGB image obtained after de-mosaicing and solving the problem of how to perform de-mosaic processing on images with an RGBW pixel structure. Attached Figure Description

[0017] Figure 1 This is a flowchart of the image demosaic method based on RGBW pixels provided in this embodiment; Figure 2 This is a schematic diagram of pixel distribution in the RGBW pixel structure provided in this embodiment; Figure 3 This is a structural block diagram of the image demosaic device based on RGBW pixels provided in this embodiment. Detailed Implementation

[0018] The image demosaic method and apparatus, electronic device, and computer storage medium based on RGBW pixels proposed in this invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the drawings are all in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of this invention. Furthermore, the structures shown in the drawings are often part of the actual structure. In particular, different proportions may be used in different drawings to illustrate different aspects.

[0019] It should be noted that the terms "first," "second," etc., used in the specification, claims, and drawings of this invention are used to distinguish similar objects in order to describe embodiments of the invention, and are not used to describe a specific order or sequence. It should be understood that such uses of terminology are interchangeable where appropriate. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0020] This embodiment provides an image demosaic method based on RGBW pixels, such as... Figure 1 As shown, it includes: S1, Obtain the original RAW image with RGBW pixel structure; S2, using the W pixel information in the original RAW image, obtain the full-resolution W brightness image; S3, using the W pixel information and W brightness image in the original RAW image, calculate the gradient information of the RGB channels; S4. Using the gradient information of the RGB channels, calculate the W pixel value of each pixel in the original RAW image to obtain the W interpolated image. S5 uses W interpolation to interpolate and enhance the RGB channels to obtain an RGB image.

[0021] The image demosaic method based on RGBW pixels provided in this embodiment calculates the gradient information of the RGB channels by introducing a W brightness image, and then interpolates the pixels to effectively preserve image details and textures. At the same time, by enhancing the RGB channels with the W interpolated image, image details and textures can be better restored, and moiré patterns and aliasing can be reduced, effectively improving the quality of the RGB image obtained after demosaicing and solving the problem of how to perform demosaic processing on images with RGBW pixel structure.

[0022] Specifically, in this embodiment, step S1 involves obtaining the original RAW image with an RGBW pixel structure.

[0023] In practical applications, image sensors with an RGBW pixel structure can be used to take pictures and obtain raw RAW images.

[0024] Furthermore, in this embodiment, step S2, the method for obtaining a full-resolution W luminance image using the W pixel information in the original RAW image, includes: S21, downsample the W channel of the original RAW image by 1 / 4 to obtain the downsampled image.

[0025] Since the W channel has the highest sampling rate in the RGBW pixel structure, obtaining the brightness of the W channel can better reflect the brightness information of the original RAW image.

[0026] Meanwhile, in this embodiment, considering that in the RGBW pixel structure, such as Figure 2 As shown, R, B, G, and W pixels are arranged in 2×2 pixel units, with each 2×2 pixel unit containing 2 W pixels and 2 pixels of the same color (R, B, and G pixels). Therefore, the W channel in the original RAW image is downsampled by 1 / 4, that is, the W pixel values ​​of the two W pixels in each 2×2 pixel unit are averaged to obtain the W pixel value for that 2×2 pixel unit. This allows for the acquisition of accurate downsampled values ​​(W pixel values) based on the characteristics of the RGBW pixel structure. By traversing the original RAW image, a downsampled image with 1 / 4 resolution is obtained.

[0027] S22, upsample the downsampled image back to its original resolution to obtain the W brightness image.

[0028] In practical applications, the downsampled image can be upsampled back to its original resolution using methods such as bilinear interpolation and bicubic interpolation to obtain a brightness image of W. Of course, other upsampling methods can also be used in other embodiments to obtain the brightness image of W, and this application does not limit this. Furthermore, the specific implementation methods of image upsampling using bilinear interpolation, bicubic interpolation, etc., are well known to those skilled in the art, and will not be elaborated upon here.

[0029] Furthermore, in this embodiment, step S3, the method for calculating the gradient information of the RGB channels using the W pixel information and W luminance image in the original RAW image, includes: S31, using the pixel value of the W pixel in the original RAW image, calculate the gradient values ​​of the RGB channels in multiple directions and at multiple frequencies.

[0030] In other words, the gradient values ​​of R, B, and G pixels in the original RAW image are calculated in multiple directions and at multiple frequencies. The directions include horizontal, vertical, and diagonal directions; the frequency represents the distance between the current pixel and the pixel used to calculate the gradient of the current pixel, where the greater the distance, the lower the frequency.

[0031] Specifically, in this embodiment, the gradient values ​​include gradient values ​​in 8 directions and 3 frequencies. The 8 directions include angles of 0°, 22.5°, 45°, 67.5°, 90°, 112.5°, 135°, and 157.5°, and the 3 frequencies include high frequency, mid frequency, and low frequency. (See also...) Figure 2Taking pixel G at coordinates (0,0) as an example, the gradient value is calculated as follows: In the horizontal direction: Low-frequency gradient:

[0032] Mid-frequency gradient:

[0033] High-frequency gradient:

[0034] The gradient calculation method for the three frequencies in the vertical direction is similar to that for the three frequencies in the horizontal direction, and will not be repeated here.

[0035] Taking the 22.5° direction as an example, diagonally upwards: Low-frequency gradient:

[0036] Mid-frequency gradient:

[0037] High-frequency gradient:

[0038] Where W(i,j) represents the W pixel value of the W pixel at coordinate (i,j).

[0039] Those skilled in the art can learn from the above examples how to calculate the oblique frequency gradients at other angles, and this application will not elaborate further.

[0040] Of course, in other embodiments, the direction angle, quantity, and frequency can be reasonably set according to actual hardware resources and accuracy requirements. Furthermore, the gradient value can be calculated using single-pixel subtraction or multi-pixel subtraction; the specific calculation method for the gradient value is well known to those skilled in the art, and this application does not limit it and will not elaborate further.

[0041] The gradient calculation method provided in this embodiment can effectively improve the difference accuracy of textures in non-horizontal and non-vertical directions by calculating gradient values ​​in multiple directions and multiple frequencies, and can also make the judgment of textures and details in each direction more accurate, thereby improving the effect of image de-mosaicing.

[0042] S32, based on the W luminance image, performs fusion processing on the gradient values ​​of multiple directions and multiple frequencies of the RGB channels to obtain the fused gradient values ​​of the RGB channels.

[0043] Because the gradient scales differ across brightness regions—for example, the visual difference of a 10-pixel gradient in a dark region may be similar to the visual difference of a 100-pixel gradient in a bright region—in this embodiment, to ensure that the gradient values ​​better reflect the overall brightness information of the image, the gradient values ​​for each direction and frequency of the RGB channels are first normalized based on the W brightness image to obtain normalized gradient values ​​for multiple directions and frequencies. Specifically, the gradient values ​​for each direction and frequency of the current pixel can be divided by the brightness of the current pixel in the W brightness image to obtain the normalized gradient values.

[0044] Then, the normalized gradient values ​​of all frequencies in the same direction of the RGB channels are weighted and fused to obtain the fused gradient value in that direction. In this way, the fused gradient value can be used to adaptively enhance the texture of various frequencies, making the gradient calculation more consistent with the currently matched texture scale. For example, low-frequency textures refer more to low-frequency gradients, and high-frequency textures refer more to high-frequency gradients.

[0045] This embodiment presents a method for weighted fusion processing of gradients:

[0046]

[0047]

[0048]

[0049] Among them, a0, b0, c0, D, and K are all self-configurable constants that can be calibrated and configured according to the usage conditions. The main considerations are scene brightness, sensor parameters, and lens characteristics. For example, when used for monitoring extremely dark scenes, a0 > b0 > c0 is configured to increase the weight of low frequencies, so as to make full use of the effective information in low frequencies and improve the reliability of the processing results. g_low, g_mid, and g_high represent the low-frequency gradient, mid-frequency gradient, and high-frequency gradient in the current direction, respectively, and g represents the fusion gradient in the current direction.

[0050] Of course, in other embodiments, other fusion processing methods may be used based on the calculation method of the fusion gradient value provided in this application, and this application does not limit such methods. Other methods of calculating the fusion gradient that do not violate the spirit of this application should also fall within the protection scope of this application.

[0051] Preferably, considering that the oblique gradient affects the oblique interpolation ratio, and in some complex texture regions, the oblique gradient is close in magnitude to the horizontal and vertical gradients, but the oblique pixels are far apart and have low correlation, even if the oblique gradient is small, the oblique pixels should not be interpolated. Therefore, in this embodiment, step S3, the method of calculating the gradient information of the RGB channels using the W pixel information and W brightness image in the original RAW image, further includes: S33 uses the horizontal and vertical fusion gradient values ​​to correct the diagonal fusion gradient value.

[0052] In this way, by correcting the oblique fusion gradient value, the oblique interpolation can be kept effective only at the more obvious oblique texture positions (such as blurred edges, moiré patterns, etc.), thereby ensuring the accuracy of the interpolation results while preserving the image texture details as much as possible, and reducing the negative effects that the oblique gradient may bring (erroneous interpolation of complex textures, such as moiré patterns, aliasing, etc.), thus improving the quality of the RGB image after mosaic removal.

[0053] In practical applications, the method for correcting the oblique fusion gradient value can be flexibly designed according to subjective and objective factors such as the sensor, lens, and usage conditions. This application does not impose any restrictions on this. This embodiment provides a method for correcting the oblique fusion gradient value, taking the above eight angular directions as examples: When the horizontal (0°) fusion gradient value or the vertical (90°) fusion gradient value is small, add a positive offset 1 to the 6 diagonal fusion gradient values; When both the horizontal and vertical fusion gradient values ​​are large, first determine the magnitudes of the fusion gradient values ​​in the 45° and 135° directions. When the fusion gradient in the 45° or 135° direction is small, add a positive offset offset2 to the remaining four diagonal fusion gradient values. When both the 45° and 135° gradients are large, calculate the absolute differences between the horizontal and vertical fusion gradient values ​​and the fusion gradient values ​​in other directions. If the absolute difference is small, add negative offsets offset3 and offset4 to the horizontal and vertical fusion gradient values, respectively. Among them, offset1, offset2, offset3 and offset4 can be used with different constant values ​​according to the scenario, or they can be calculated by designing a model based on the magnitude of the gradient.

[0054] Specifically, let the fusion gradient values ​​for the eight directions be g0, g22.5, g45, g67.5, g90, g112.5, g135, and g157.5, and set as follows:

[0055]

[0056]

[0057]

[0058]

[0059]

[0060]

[0061]

[0062] in, It is a function that is monotonically decreasing with respect to x. ; It is a function that is monotonically decreasing with respect to y. ; It is about A function that is monotonically increasing and not greater than 0, when When large enough, TH1 and TH2 are preset threshold values.

[0063] Furthermore, in this embodiment, step S4, which uses the gradient information of the RGB channels to calculate the W pixel value of each pixel in the original RAW image to obtain the W interpolated image, includes the following method: S41, using the gradient information of the RGB channels, calculate the W interpolation weights.

[0064] In practical applications, the inverse of the gradient value obtained in step S3 can be normalized and used as the W interpolation weight. Alternatively, a monotonically decreasing function corresponding to the gradient value can be set for the W interpolation weight according to actual needs. The specific methods for obtaining the interpolation weight using gradient information are well known to those skilled in the art, and will not be elaborated here.

[0065] S42 calculates the W pixel value of RGB pixels using the W interpolation weight.

[0066] For R, B, and G pixels, the corresponding W pixel values ​​need to be obtained by interpolating the W pixel. To better reflect local image details, in this embodiment, the W interpolation weight is used to perform weighted interpolation calculations on the R, B, and G pixels to obtain the corresponding W pixel values. The process of weighted interpolation calculation for R, B, and G pixels can be represented as follows:

[0067] Where weight represents the W interpolation weight corresponding to that direction; W represents the W pixel interpolation in that direction, for example, Figure 2 For pixel G at coordinate (0,0), the horizontal W pixel interpolation W0 can be: W0=(W(0,-1)+W(0,1)) / 2; and the 22.5° W pixel interpolation W22.5 can be: W22.5=(W(1,2)+W(-1,-2)) / 2.

[0068] Those skilled in the art can learn how to calculate W-pixel interpolation in other directions based on the above-described method, and this application will not elaborate on these methods further. Furthermore, in other embodiments, W-pixel interpolation in various directions may also be calculated in other ways. The calculation of W-pixel interpolation is well-known to those skilled in the art, and this application does not impose any restrictions on it.

[0069] S43, take the original pixel value of pixel W as the W pixel value of pixel W.

[0070] S44 combines the W pixel value of the RGB pixel and the W pixel value of the W pixel to obtain the W interpolated image.

[0071] Furthermore, in this embodiment, step S5, the method of interpolating and enhancing the RGB channels using the W interpolated image to obtain an RGB image, includes: S51, calculate the energy intensity gain of the original RAW image.

[0072] Specifically, in this embodiment, the arithmetic mean of the four RGBW channels of the original RAW image is first calculated, denoted as MeanR, MeanB, MeanG, and MeanW respectively; then, the energy intensity gain TimesW is calculated using the arithmetic mean of the four channels, wherein the calculation formula is:

[0073] S52, multiply the W interpolated image by the energy intensity gain to obtain the W enhanced image. This process can be represented as:

[0074] Where W represents the W-interpolated image, and W_T represents the W-enhanced image.

[0075] S53, downsample the RGB channels in the W enhanced image and the original RAW image by 1 / 16 to obtain the W downsampled image, R downsampled image, G downsampled image and B downsampled image.

[0076] Considering the RGBW pixel structure, such as Figure 2As shown, a 4×4 pixel area constitutes a complete RGBW unit, which includes a 2×2 pixel range containing a group of R pixels, a 2×2 pixel range containing a group of B pixels, and two 2×2 pixel ranges containing two groups of G pixels. Therefore, in this embodiment, the RGB channels in the W-enhanced image and the original RAW image are downsampled by 1 / 16, which not only ensures that the downsampled image obtains accurate and complete image information, but also preserves as many image details as possible, and also corresponds to the RGB format.

[0077] The downsampling method used here can be similar to the downsampling method in step S21, and will not be described in detail here.

[0078] S54, subtract the W downsampled image from the R downsampled image, the G downsampled image, and the B downsampled image respectively to obtain the WR downsampled difference image, the WG downsampled difference image, and the WB downsampled difference image.

[0079] The specific implementation of this step is similar to the method of obtaining the color difference image by subtraction in the RGB pixel de-mosaic method, and will not be described in detail here. At this time, the resolution of the obtained WR downsampled difference image, WG downsampled difference image and WB downsampled difference image is 1 / 16 of the resolution of the original RAW image.

[0080] S55, upsample the WR downsampled difference image, WG downsampled difference image, and WB downsampled difference image back to their original resolution to obtain the WR difference image, WG difference image, and WB difference image.

[0081] The upsampling method used in this step can be similar to that used in step S22, and will not be described in detail here.

[0082] S56, the W-enhanced image is subtracted from the WR difference image, WG difference image and WB difference image respectively to obtain the first R image, the first G image and the first B image.

[0083] The specific implementation of this step is similar to the method of obtaining an RGB monochrome image by subtraction in the RGB pixel demosaic method, and will not be described in detail here.

[0084] S57, combine the first R image, the first G image and the first B image to obtain an RGB image.

[0085] The specific implementation of this step is similar to the method of combining images to obtain an RGB image in the RGB pixel demosaic method, and will not be described in detail here.

[0086] Preferably, in order to make the obtained RGB image smoother, in this embodiment, step S5, the method of interpolating and enhancing the RGB channels using the W interpolated image to obtain the RGB image further includes: S58-1, Update the R pixel values ​​in the first R image to obtain the second R image, wherein: for an R pixel in the original RAW image, calculate the difference between its window filter value in the first R image and its R pixel value in the original RAW image; if the difference is greater than a preset R difference threshold, then use the R pixel value in the first R image as the R pixel value of that R pixel; otherwise, perform weighted fusion on the R pixel values ​​in the first R image and the R pixel values ​​in the original RAW image to obtain the updated R pixel value of that R pixel. S58-2, Update the G pixel values ​​in the first G image to obtain the second G image, wherein: for a G pixel in the original RAW image, calculate the difference between its window filter value in the first G image and its G pixel value in the original RAW image; if the difference is greater than a preset G difference threshold, then use the G pixel value in the first G image as the G pixel value of that G pixel; otherwise, perform weighted fusion on the G pixel values ​​in the first G image and the G pixel values ​​in the original RAW image to obtain the updated G pixel value of that G pixel. S58-3, Update the B pixel values ​​in the first B image to obtain the second B image, wherein: for a B pixel in the original RAW image, calculate the difference between its window filter value in the first B image and its B pixel value in the original RAW image; if the difference is greater than a preset B difference threshold, then use the B pixel value in the first B image as the B pixel value of that B pixel; otherwise, perform weighted fusion on the B pixel values ​​in the first B image and the B pixel values ​​in the original RAW image to obtain the updated B pixel value of that B pixel. The R-interference threshold, G-interference threshold, and B-interference threshold can be set reasonably according to actual needs. Furthermore, the window size corresponding to the window filter value can also be set reasonably according to actual needs. Specific filtering methods can include Gaussian filtering, mean filtering, etc., and this application does not impose any restrictions on this.

[0087] S59, combine the second R image, the second G image, and the second B image to obtain an RGB image.

[0088] The image demosaic method based on RGBW pixels provided in this embodiment combines brightness information to normalize gradient values, making the gradients more accurate. Simultaneously, by weighted fusion and correction of gradients at different frequencies, it can adaptively determine textures of different frequencies with greater precision. This ensures accurate texture details in the resulting RGB image after demosaicing, reduces moiré patterns and aliasing, and improves image quality.

[0089] The image demosaic method based on RGBW pixels provided in this embodiment calculates the energy intensity gain and combines it with the W interpolated image to complete the interpolation and enhancement processing of the RGB channels. This can make up for the shortcomings of low sampling rate and less effective information in the RGB channels, make full use of the W channel information to achieve image enhancement, better restore image details and textures, and ensure the quality of RGB images.

[0090] This embodiment also provides an image demosaic device based on RGBW pixels, used to implement the image demosaic method based on RGBW pixels as described above, such as... Figure 3 As shown, the image demosaic device based on RGBW pixels includes: The image acquisition module is used to acquire the original RAW image with RGBW pixel structure; The brightness calculation module is used to obtain a full-resolution W brightness image by utilizing the W pixel information in the original RAW image; The gradient calculation module is used to calculate the gradient information of the RGB channels using the W pixel information and W brightness image in the original RAW image. The interpolation processing module is used to calculate the W pixel value of each pixel in the original RAW image using the gradient information of the RGB channels, so as to obtain the W interpolated image. The enhancement processing module is used to interpolate and enhance the RGB channels using the W interpolated image to obtain an RGB image.

[0091] The image de-mosaic device based on RGBW pixels provided in this embodiment calculates the gradient information of the RGB channels by introducing a W brightness image, and then interpolates the pixels so that the image details and textures can be effectively preserved. At the same time, the RGB channels are enhanced by the W interpolated image, which can better restore the image details and textures and reduce moiré patterns, aliasing, etc., effectively improving the quality of the RGB image obtained by de-mosaicing and solving the problem of how to perform de-mosaic processing on images with RGBW pixel structure.

[0092] Furthermore, this embodiment also provides an electronic device, including a memory, a processor, and an executable program stored in the memory and capable of being run by the processor; when the processor runs the executable program, it performs the image de-mosaic method based on RGBW pixels as described above.

[0093] Furthermore, this embodiment also provides a computer storage medium storing an executable program; when the executable program is executed, it implements the image demosaic method based on RGBW pixels as described above.

[0094] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to mutually. In addition, different parts between embodiments can also be combined with each other, and this invention does not limit this.

[0095] This embodiment provides an image de-mosaic method, apparatus, electronic device, and computer storage medium based on RGBW pixels. The method includes: acquiring an original RAW image with an RGBW pixel structure; obtaining a full-resolution W luminance image using the W pixel information in the original RAW image; calculating the gradient information of the RGB channels using the W pixel information and the W luminance image in the original RAW image; calculating the W pixel value of each pixel in the original RAW image using the gradient information of the RGB channels to obtain a W interpolated image; and performing interpolation and enhancement processing on the RGB channels using the W interpolated image to obtain an RGB image. By introducing the W luminance image to calculate the gradient information of the RGB channels and then interpolating the pixels, image details and textures can be effectively preserved. Simultaneously, enhancing the RGB channels using the W interpolated image better restores image details and textures and reduces moiré patterns and aliasing, effectively improving the quality of the RGB image obtained after de-mosaicing and solving the problem of how to perform de-mosaic processing on images with an RGBW pixel structure.

[0096] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the present invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the claims.

Claims

1. A method for demosaicing images based on RGBW pixels, characterized in that, include: Obtain the original RAW image with RGBW pixel structure; By utilizing the W pixel information in the original RAW image, a full-resolution W brightness image is obtained; Calculate the gradient information of the RGB channels using the W pixel information and W brightness image in the original RAW image; Using the gradient information of the RGB channels, the W pixel value of each pixel in the original RAW image is calculated to obtain the W interpolated image. The W interpolation image is used to interpolate and enhance the RGB channels to obtain an RGB image.

2. The image demosaic method based on RGBW pixels according to claim 1, characterized in that, The method for obtaining a full-resolution W luminance image using the W pixel information in the original RAW image includes: The W channel of the original RAW image is downsampled by 1 / 4 to obtain a downsampled image; The downsampled image is upsampled back to its original resolution to obtain the W brightness image.

3. The image demosaic method based on RGBW pixels according to claim 1, characterized in that, The method for calculating the gradient information of the RGB channels using the W pixel information and W luminance image in the original RAW image includes: Using the pixel value of pixel W in the original RAW image, the gradient values ​​of the RGB channels in multiple directions and at multiple frequencies are calculated; the directions include the horizontal direction, the vertical direction, and the diagonal direction; the frequency represents the distance between the current pixel and the pixel used to calculate the gradient of the current pixel, wherein the greater the distance, the lower the frequency. Based on the W luminance image, the gradient values ​​of multiple directions and multiple frequencies of the RGB channels are fused to obtain the fused gradient values ​​of the RGB channels.

4. The image demosaic method based on RGBW pixels according to claim 3, characterized in that, The method for fusing gradient values ​​of multiple directions and multiple frequencies of the RGB channels based on the W luminance image to obtain the fused gradient values ​​of the RGB channels includes: Based on the W luminance image, the gradient values ​​of each direction and each frequency of the RGB channel are normalized to obtain normalized gradient values ​​of multiple directions and multiple frequencies. The normalized gradient values ​​of all frequencies in the same direction of the RGB channels are weighted and fused to obtain the fused gradient value in that direction.

5. The image demosaic method based on RGBW pixels according to claim 3, characterized in that, The method for calculating the gradient information of the RGB channels using the W pixel information and W brightness image in the original RAW image further includes: The horizontal and vertical fusion gradient values ​​are used to correct the diagonal fusion gradient value.

6. The image demosaic method based on RGBW pixels according to claim 1, characterized in that, The method for calculating the W pixel value of each pixel in the original RAW image using gradient information from the RGB channels to obtain the W interpolated image includes: Calculate the W interpolation weights using the gradient information of the RGB channels; Calculate the W pixel value of the RGB pixel using the W interpolation weight; Use the original pixel value of pixel W as the W pixel value of pixel W; Combine the W pixel values ​​of the RGB pixels and the W pixel values ​​of the W pixels to obtain the W interpolated image.

7. The image demosaic method based on RGBW pixels according to claim 1, characterized in that, The method of interpolating and enhancing the RGB channels using W interpolation to obtain an RGB image includes: Calculate the energy intensity gain of the original RAW image; The W interpolated image is multiplied by the energy intensity gain to obtain the W enhanced image; The RGB channels in the W-enhanced image and the original RAW image are downsampled by 1 / 16 to obtain the W-downsampled image, R-downsampled image, G-downsampled image and B-downsampled image; The W-downsampled image is subtracted from the R-downsampled image, G-downsampled image, and B-downsampled image respectively to obtain the WR-downsampled difference image, WG-downsampled difference image, and WB-downsampled difference image. Upsample the WR downsampled difference image, WG downsampled difference image, and WB downsampled difference image back to their original resolution to obtain the WR difference image, WG difference image, and WB difference image; The W-enhanced image is subtracted from the WR difference image, WG difference image, and WB difference image respectively to obtain the first R image, the first G image, and the first B image; Combine the first R image, the first G image, and the first B image to obtain an RGB image.

8. The image demosaic method based on RGBW pixels according to claim 7, characterized in that, After obtaining the first R image, the first G image, and the first B image, the method of interpolating and enhancing the RGB channels using W interpolation to obtain an RGB image further includes: The R pixel values ​​in the first R image are updated to obtain the second R image, wherein: for an R pixel in the original RAW image, the difference between its window filter value in the first R image and its R pixel value in the original RAW image is calculated; if the difference is greater than a preset R difference threshold, the R pixel value in the first R image is used as the R pixel value of that R pixel; otherwise, the R pixel values ​​in the first R image and the R pixel values ​​in the original RAW image are weighted and fused to obtain the updated R pixel value of that R pixel. The G-pixel values ​​in the first G-image are updated to obtain the second G-image, wherein: for a G-pixel in the original RAW image, the difference between its window filter value in the first G-image and its G-pixel value in the original RAW image is calculated; if the difference is greater than a preset G difference threshold, the G-pixel value in the first G-image is used as the G-pixel value of that G-pixel; otherwise, the G-pixel values ​​in the first G-image and the G-pixel values ​​in the original RAW image are weighted and fused to obtain the updated G-pixel value of that G-pixel. The B pixel values ​​in the first B image are updated to obtain the second B image, wherein: for a B pixel in the original RAW image, the difference between its window filter value in the first B image and its B pixel value in the original RAW image is calculated; if the difference is greater than a preset B difference threshold, the B pixel value in the first B image is used as the B pixel value of that B pixel; otherwise, the B pixel values ​​in the first B image and the B pixel values ​​in the original RAW image are weighted and fused to obtain the updated B pixel value of that B pixel. Combine the second R image, the second G image, and the second B image to obtain an RGB image.

9. An image demosaic device based on RGBW pixels, used to implement the image demosaic method based on RGBW pixels as described in any one of claims 1 to 8, characterized in that, The image demosaic device based on RGBW pixels includes: The image acquisition module is used to acquire the original RAW image with RGBW pixel structure; The brightness calculation module is used to obtain a full-resolution W brightness image by utilizing the W pixel information in the original RAW image; The gradient calculation module is used to calculate the gradient information of the RGB channels using the W pixel information and W brightness image in the original RAW image. The interpolation processing module is used to calculate the W pixel value of each pixel in the original RAW image using the gradient information of the RGB channels, so as to obtain the W interpolated image. The enhancement processing module is used to interpolate and enhance the RGB channels using the W interpolated image to obtain an RGB image.

10. An electronic device, characterized in that, It includes a memory, a processor, and an executable program stored in the memory and capable of being run by the processor; when the processor runs the executable program, it performs the image demosaic method based on RGBW pixels as described in any one of claims 1 to 8.

11. A computer storage medium, characterized in that, The computer storage medium stores an executable program; when the executable program is executed, it implements the image demosaic method based on RGBW pixels as described in any one of claims 1 to 8.

Images