RGBW image-based color restoration method and device, electronic equipment and computer storage medium
By obtaining the mapping matrix and color correction matrix of RGBW images, calculating the estimated value of W pixels and the probability of gray pixels, and performing automatic white balance and color correction, the signal-to-noise ratio and color accuracy problems in RGBW image color restoration are solved, achieving high signal-to-noise ratio and accurate RGB image restoration.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHENGDU LIGHT COLLECTOR TECH
- Filing Date
- 2026-01-30
- Publication Date
- 2026-05-01
AI Technical Summary
Existing technologies struggle to effectively utilize the high signal-to-noise ratio (SNR) W pixel signals of RGBW sensors to enhance image brightness while maintaining the accuracy of RGB pixel color information, resulting in poor color reproduction in RGBW images.
By obtaining the mapping matrix between the RGB response curve and the W response curve, and the four-channel color correction matrix, the estimated value of W pixels and the probability of gray pixels are calculated, automatic white balance correction and color correction are performed, and image restoration is performed using the W pixel information.
It achieves more accurate and stable automatic white balance processing, improves the signal-to-noise ratio and color accuracy of the image, and obtains RGB images with high signal-to-noise ratio.
Smart Images
Figure CN121967652A_ABST
Abstract
Description
Color restoration methods and apparatus, electronic devices and computer storage media based on RGBW images Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to a color restoration method and apparatus, electronic device and computer storage medium based on RGBW images. Background Technology
[0002] With the development of display technology, RGBW display technology was proposed. RGBW adds a white subpixel to the traditional RGB base to make the image colors richer and the color gamut wider.
[0003] While RGBW sensors possess the potential for high sensitivity in hardware, their ability to translate the acquired images into a high-quality final image heavily relies on the efficiency of the backend image signal processing workflow. Currently, image signal processing for RGBW sensors is typically based on RGB, requiring the interpolation and reconstruction of a full-color image where each pixel contains complete RGB information from the sparse raw RGBW data. A significant challenge in this conversion process is how to fully utilize the high signal-to-noise ratio (SNR) W pixel signals to enhance the overall image's brightness information without compromising the accuracy of the color information determined by the low SNR RGB pixels. Summary of the Invention
[0004] The purpose of this invention is to provide a color restoration method, apparatus, electronic device, and computer storage medium based on RGBW images, so as to solve the problem of how to restore the color of RGBW images to obtain RGB images with high signal-to-noise ratio and accurate color.
[0005] To address the aforementioned technical problems, this invention provides a color restoration method based on RGBW images, comprising: obtaining a mapping matrix between the RGB response curve and the W response curve, and a four-channel color correction matrix at different saturations; inputting an original RGBW image and obtaining its full-resolution R image, full-resolution G image, full-resolution B image, and full-resolution W image; calculating W pixel estimates using the mapping matrix and the full-resolution R, G, and B images; calculating the probability that all pixels in the original RGBW image are gray pixels based on the full-resolution W image and the W pixel estimates; performing automatic white balance correction on the RGB channels using the probability that all pixels in the original RGBW image are gray pixels to obtain an automatically white balance corrected RGBW image; and performing color correction on the automatically white balance corrected RGBW image using the four-channel color correction matrix to obtain a color-restored RGB image.
[0006] Optionally, in the color restoration method based on RGBW images, the method for obtaining the mapping relationship matrix between the RGB response curve and the W response curve includes: obtaining multiple sets of pixel arrays based on color cards photographed under light sources of different color temperatures, each set of pixel arrays including R pixel values, G pixel values, B pixel values and W pixel values; linearly fitting the R pixel values, G pixel values, B pixel values and W pixel values to obtain the mapping relationship between the RGB response curve and the W response curve; and minimizing the mapping relationship to obtain the mapping relationship matrix between the RGB response curve and the W response curve.
[0007] Optionally, in the color restoration method based on RGBW images, the method for obtaining the four-channel color correction matrix under different saturations includes: obtaining the RGB color matrix of the color chart; using the RGB color matrix, calculating the linear standard RGB matrix with respect to saturation; photographing the color chart to obtain the RGBW color matrix; and performing a minimization objective solution on the linear standard RGB matrix and the RGBW color matrix to obtain the four-channel color correction matrix under different saturations.
[0008] Optionally, in the color restoration method based on the RGBW image, the method for calculating the probability that all pixels in the original RGBW image are gray pixels based on the full-resolution W image and the estimated W pixel value includes: calculating the error value between the W pixel value of the current pixel in the full-resolution W image and the estimated W pixel value; calculating the probability that the current pixel is a gray pixel based on the error value; and traversing the full-resolution W image to obtain the probability that all pixels are gray pixels.
[0009] Optionally, in the aforementioned color restoration method based on RGBW images, the method for automatically correcting the RGB channels using the probability that all pixels in the original RGBW image are gray pixels includes: determining whether the original RGBW image was captured under extreme color temperature light sources; if the original RGBW image was captured under extreme color temperature light sources, then no automatic white balance correction is performed, or an existing automatic white balance correction method is used; if the original RGBW image was not captured under extreme color temperature light sources, then the probability that all pixels in the original RGBW image are gray pixels is used to calculate the R channel gain value, G channel gain value, and B channel gain value, and the full-resolution R image is multiplied by the R channel gain value, the full-resolution G image is multiplied by the G channel gain value, and the full-resolution B image is multiplied by the B channel gain value to obtain the RGB channel values after automatic white balance correction.
[0010] Optionally, in the color restoration method based on RGBW images, the method for determining whether the original RGBW image was captured under an extreme color temperature light source includes: calculating the probability sum of pixels that are gray pixels in the original RGBW image; if the probability sum is less than a probability threshold, then the original RGBW image is determined to have been captured under an extreme color temperature light source, otherwise it is not captured under an extreme color temperature light source.
[0011] Optionally, in the color restoration method based on RGBW images, the method of using a four-channel color correction matrix to perform color correction on the RGBW image after automatic white balance correction to obtain a color-restored RGB image includes: interpolating the four-channel color correction matrix according to the desired saturation to obtain a target color correction matrix; and using the target color correction matrix to perform color correction on the RGBW image after automatic white balance correction to obtain a color-restored RGB image.
[0012] To address the aforementioned technical problems, this invention also provides a color restoration device based on RGBW images, used to implement the color restoration method based on RGBW images as described in any of the preceding claims. The color restoration device based on RGBW images includes: an offline acquisition module for acquiring the mapping relationship matrix between the RGB response curve and the W response curve, and a four-channel color correction matrix at different saturations; an online acquisition module for acquiring the original RGBW image, and acquiring the full-resolution R image, full-resolution G image, full-resolution B image, and full-resolution W image of the original RGBW image; an estimation calculation module for calculating the estimated value of W pixels using the mapping relationship matrix and the full-resolution R image, full-resolution G image, and full-resolution B image; a probability calculation module for calculating the probability that all pixels in the original RGBW image are gray pixels based on the full-resolution W image and the estimated value of W pixels; a white balance correction module for automatically white-balancing the RGB channels using the probability that all pixels in the original RGBW image are gray pixels, to obtain an automatically white-balanced RGBW image; and a color correction module for color correction of the automatically white-balanced RGBW image using the four-channel color correction matrix, to obtain a color-restored RGB image.
[0013] To address the aforementioned technical problems, the present invention also provides an electronic device, including a memory, a processor, and an executable program stored in the memory and capable of being run by the processor; when the processor runs the executable program, it executes the color restoration method based on RGBW images as described in any of the preceding claims.
[0014] To address the aforementioned technical problems, the present invention also provides a computer storage medium storing an executable program; when the executable program is executed, it implements the color restoration method based on RGBW images as described in any of the preceding claims.
[0015] The present invention provides a color restoration method, apparatus, electronic device, and computer storage medium based on RGBW images, comprising: obtaining a mapping matrix between the RGB response curve and the W response curve and a four-channel color correction matrix at different saturations; inputting an original RGBW image and obtaining a full-resolution R image, a full-resolution G image, a full-resolution B image, and a full-resolution W image of the original RGBW image; calculating W pixel estimates using the mapping matrix and the full-resolution R, G, and B images; calculating the probability that all pixels in the original RGBW image are gray pixels based on the full-resolution W image and the W pixel estimates; performing automatic white balance correction on the RGB channels using the probability that all pixels in the original RGBW image are gray pixels to obtain an automatically white balance corrected RGBW image; and performing color correction on the automatically white balance corrected RGBW image using the four-channel color correction matrix to obtain a color-restored RGB image. By obtaining the mapping relationship between the RGB channels and the W channels and calculating the estimated value of the W pixels to obtain the probability of gray pixels, and then using this probability to perform automatic white balance correction, more accurate and stable automatic white balance processing of the image can be achieved. By performing color correction processing through a four-channel color correction matrix, the color correction process can utilize the W pixel information to obtain a better signal-to-noise ratio and more accurate color, solving the problem of how to perform color restoration on RGBW images to obtain RGB images with high signal-to-noise ratio and accurate color. Attached Figure Description
[0016] Figure 1 is a flowchart of the color restoration method based on RGBW images provided in this embodiment; Figure 2 is a schematic diagram of the structure of the color restoration device based on RGBW images provided in this embodiment. Detailed Implementation
[0017] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, provides a more comprehensive overview of the color restoration method, apparatus, electronic device, and computer storage medium based on RGBW images proposed in this invention. It should be noted that the accompanying drawings are all in a very simplified form and use non-precise proportions, intended only to facilitate and clarify the illustration of the embodiments of this invention. Furthermore, the structures shown in the drawings are often part of the actual structures. In particular, different proportions may be used in different drawings to emphasize different aspects.
[0018] It should be noted that the terms "first," "second," etc., used in the specification, claims, and drawings of this invention are used to distinguish similar objects in order to describe embodiments of the invention, and are not used to describe a specific order or sequence. It should be understood that such uses of terminology are interchangeable where appropriate. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0019] The color reproduction process mainly includes Automatic White Balance (AWB) and Color Correct Matrix (CCM). Automatic White Balance aims to approximate the color constancy of the human visual system, that is, the ability to perceive the color of an object as roughly constant regardless of the color of the light source. Color Correction converts the image from a device-dependent color space to a device-independent color space, typically the sRGB color space.
[0020] This embodiment provides a color restoration method based on RGBW images, as shown in Figure 1, including: S1, obtaining the mapping relationship matrix between the RGB response curve and the W response curve and the four-channel color correction matrix under different saturations; S2, inputting the original RGBW image and obtaining the full-resolution R image, full-resolution G image, full-resolution B image and full-resolution W image of the original RGBW image; S3, using the mapping relationship matrix and the full-resolution R image, full-resolution G image and full-resolution B image, calculating the estimated value of W pixels; S4, based on the full-resolution W image and the estimated value of W pixels, calculating the probability that all pixels in the original RGBW image are gray pixels; S5, using the probability that all pixels in the original RGBW image are gray pixels, performing automatic white balance correction on the RGB channels to obtain the automatically white balance corrected RGBW image; S6, using the four-channel color correction matrix, performing color correction on the automatically white balance corrected RGBW image to obtain the color-restored RGB image.
[0021] The color restoration method based on RGBW images provided in this embodiment obtains the mapping relationship between the RGB channels and the W channels and calculates the estimated value of the W pixels to obtain the probability of gray pixels. Then, it uses this probability to perform automatic white balance correction, which can complete more accurate and stable automatic white balance processing of the image. By performing color correction processing through a four-channel color correction matrix, the color correction process can use the W pixel information to obtain a better signal-to-noise ratio and more accurate color, thus solving the problem of how to restore the color of RGBW images to obtain RGB images with high signal-to-noise ratio and accurate color.
[0022] Specifically, in this embodiment, step S1 can be performed offline, and the obtained mapping matrix and four-channel color correction matrix are stored for subsequent steps to call.
[0023] Furthermore, in this embodiment, step S1 involves obtaining the mapping matrix between the RGB response curve and the W response curve, and the four-channel color correction matrix under different saturations.
[0024] The method for obtaining the mapping matrix between the RGB response curve and the W response curve includes: First, based on color charts photographed under light sources of different color temperatures, multiple pixel arrays are obtained. Each pixel array includes R pixel values, G pixel values, B pixel values, and W pixel values. In a specific embodiment, 24 color charts can be photographed under N different color temperature light sources, where N is a positive integer greater than 1. Thus, N×24 pixel arrays are obtained. Each pixel array contains one R pixel value, one G pixel value, one B pixel value, and one W pixel value corresponding to one color in the color chart at one color temperature. The pixel array is denoted as [SR, SG, SB, SW]. The pixel arrays [SR, SG, SB, SW] are all matrices of size [24N, 1].
[0025] Next, considering that the response curve of the W channel is approximately a horizontal line, covering the range of 400nm to 700nm; and that the peak values of the RGB channel response curves are at 600nm, 550nm, and 450nm respectively, with the response curves gradually decreasing on both sides of the peak, and that the combination of RGB channel response curves also covers the range of 400nm to 700nm, the response curve of the W channel can be approximated by a linear combination of the three RGB channel response curves. That is, by linearly fitting the R pixel value, G pixel value, B pixel value, and W pixel value, the mapping relationship between the RGB response curve and the W response curve can be obtained, expressed as:
[0026] Where m represents the mapping relationship matrix.
[0027] Finally, the mapping relationship is minimized to obtain the mapping matrix between the RGB response curve and the W response curve. Specifically, in this embodiment, the least squares method is used to minimize the objective, as shown below:
[0028] The argmin() function is used to find the value of the independent variable that minimizes the objective function (or array).
[0029] Solving the above equation yields the mapping matrix, which is expressed as:
[0030] The method provided in this embodiment, which obtains a mapping matrix by fitting the RGB channel with the W channel, can have a smaller error when calculating gray pixels in the subsequent process, thereby helping to perform more accurate and stable automatic white balance processing on the image during automatic white balance correction.
[0031] Furthermore, the method for obtaining the four-channel color correction matrix at different saturations includes: first, obtaining the RGB color matrix of the color chart, denoted as [CR,CG,CB]. In practical applications, the RGB color matrix of a 24-color chart is also obtained. The specific methods for obtaining the RGB color matrix are well known to those skilled in the art, and will not be elaborated upon here.
[0032] Then, using the RGB color matrix, a linear standard RGB matrix with respect to saturation is calculated. Specifically, the expected values of standard colors in a 24-color chart at different saturations are calculated. First, the RGB color matrix is converted to HSV, represented as:
[0033] When the saturation is k, we can obtain:
[0034] Converting HSV with saturation of k back to RGB is represented as:
[0035] Convert the RGB color matrix containing saturation k information into a linear RGB color matrix to obtain a linear standard RGB matrix, represented as:
[0036] Thus, by changing the saturation level k, we can obtain the linear standard RGB matrix corresponding to different saturations.
[0037] Next, the color chart is photographed to obtain the RGBW color matrix. Specifically, the same color chart, such as a 24-color chart, is photographed using an RGBW image sensor to obtain RGBW color chart images. These RGBW color chart images are then reconstructed into full-resolution R, G, B, and W images, and automatic white balance correction is performed to obtain a four-channel image of the color chart, denoted as... Then, extract the 4-channel color values of 24 standard colors from the color patch positions of the 24-color chart to obtain a 24-color matrix, denoted as... .
[0038] Finally, the objective function of minimizing the linear standard RGB matrix and the RGBW color matrix is performed to obtain the four-channel color correction matrix under different saturations. Specifically, in this embodiment, the least squares method is used to calculate the four-channel color correction matrix under different saturations, as follows:
[0039] In practical applications, the timing of obtaining the four-channel color correction matrix can be performed in step S5, after automatic white balance correction is completed; this application does not impose any restrictions on this. Furthermore, in practical applications, the specific method for obtaining the RGBW color matrix can be implemented using steps S2 to S5 of this application, that is, the captured color chart image is used as the original RGBW image and processed in steps S2 to S5 until automatic white balance correction is completed, thereby obtaining the four-channel image and the color matrix.
[0040] Furthermore, in this embodiment, step S2 involves inputting the original RGBW image and obtaining the full-resolution R image, full-resolution G image, full-resolution B image, and full-resolution W image of the original RGBW image.
[0041] The specific methods for obtaining full-resolution R images, full-resolution G images, full-resolution B images, and full-resolution W images are well known to those skilled in the art, and will not be described in detail here.
[0042] Furthermore, in this embodiment, step S3 involves calculating the estimated value of pixel W using the mapping matrix and the full-resolution R image, full-resolution G image, and full-resolution B image.
[0043] Specifically, in this embodiment, the calculation method for the W pixel estimate can be expressed as follows:
[0044] in, This indicates the coordinates of the current pixel in the RGBW image. This represents the pixel value of the current pixel in the full-resolution R image. This represents the pixel value of the current pixel in the full-resolution G image. This represents the pixel value of the current pixel in a full-resolution B-image. , , and To solve for the values in the obtained mapping matrix, This represents the W-pixel estimate of the current pixel.
[0045] Furthermore, in this embodiment, step S4 involves calculating the probability that all pixels in the original RGBW image are gray pixels based on the full-resolution W image and the estimated W pixel value.
[0046] Specifically, in this embodiment, firstly, the error value between the current pixel's W pixel value and the estimated W pixel value in the full-resolution W image is calculated. The formula for calculating the error value can be expressed as:
[0047] in, This represents the pixel value of the current pixel in the full-resolution W image. This represents the estimated W pixel value calculated in step S3. This indicates the error value.
[0048] Then, based on the error value, the probability that the current pixel is a gray pixel is calculated. First, the adaptive threshold for gray pixels is calculated, expressed as:
[0049] in, This represents the W pixel value of the current pixel. This represents the estimated W pixel value obtained in step S3, min() indicates the minimum value calculation, and sqrt() indicates the square root calculation. This indicates an adaptive threshold for gray pixels.
[0050] Next, the probability that the current pixel is a gray pixel is calculated using the gray pixel adaptive threshold and the error value, expressed as:
[0051]
[0052] Where c represents an adjustable parameter, the larger the value, the more pixels are identified as gray pixels; This indicates the probability that the current pixel is a gray pixel.
[0053] Finally, iterate through the full-resolution W image to obtain the probability that all pixels are gray pixels.
[0054] Furthermore, in this embodiment, in step S5, the probability that all pixels in the original RGBW image are gray pixels is used to perform automatic white balance correction on the RGB channels to obtain an RGBW image after automatic white balance correction.
[0055] Specifically, in this embodiment, when performing automatic white balance correction, it is necessary to first determine whether the original RGBW image was captured under extreme color temperature light sources. Extreme color temperature light sources refer to light sources with color temperatures far below 3000K or far above 6500K, i.e., light sources at the extremes of the color temperature spectrum with very significant color shifts.
[0056] This embodiment provides a method for determining whether an original RGBW image was captured under extreme color temperature light sources: Calculate the sum of probabilities that pixels in the original RGBW image are gray pixels; that is, calculate the probability that each pixel in the original RGBW image is a gray pixel, and sum the probabilities corresponding to all pixels, expressed as:
[0057] If the sum of probabilities is less than the probability threshold ,Right now If the probability threshold is not specified, the original RGBW image is determined to have been captured under extreme color temperature lighting conditions; otherwise, it is not. The probability threshold is an adjustable parameter; a higher probability threshold makes it easier to determine if the current shooting environment is an extreme color temperature lighting scene.
[0058] If the original RGBW image was taken under extreme color temperature light sources, it indicates that the image does not contain the assumption of gray pixels in the scene required by the automatic white balance algorithm. In this case, the automatic white balance algorithm fails, so automatic white balance correction can be omitted, or existing automatic white balance correction methods can be used for automatic white balance correction.
[0059] If the original RGBW image was not captured under extreme color temperature lighting conditions, then first use the probability that all pixels in the original RGBW image are gray pixels to calculate the average RGB channel value of the gray pixels, expressed as:
[0060]
[0061]
[0062] Then, the R-channel gain is calculated using the R-channel mean, the G-channel gain is calculated using the G-channel mean, and the B-channel gain is calculated using the B-channel mean. The full-resolution R-image is then multiplied by the R-channel gain, the full-resolution G-image by the G-channel gain, and the full-resolution B-image by the B-channel gain to obtain the RGB channel values after automatic white balance correction, expressed as:
[0063]
[0064]
[0065] Furthermore, in this embodiment, in step S6, a four-channel color correction matrix is used to perform color correction on the RGBW image after automatic white balance correction to obtain a color-restored RGB image.
[0066] Specifically, in this embodiment, firstly, the four-channel color correction matrix is interpolated according to the desired saturation to obtain the target color correction matrix. In one specific embodiment, it is assumed that three sets of four-channel color correction matrices with saturation values of 0.1, 0.5, and 1 are calibrated, denoted as... , and The current saturation is k, where 0.5 < k < 1. Then the target color correction matrix can be expressed as:
[0067] Of course, in practical applications, those skilled in the art can learn how to obtain the target color correction matrix based on the above examples, and this application will not elaborate on this further.
[0068] Then, the target color correction matrix is used to perform color correction on the RGBW image after automatic white balance correction to obtain the color-restored RGB image, represented as:
[0069] in, Represents the target color correction matrix. This represents the RGB channel matrix.
[0070] It should be noted that the CCM target color correction matrix of this application is 4×3, which is different from the existing 3×3 CCM matrix. The three additional parameters represent the influence of the W pixel on the final output RGB channel values.
[0071] This embodiment incorporates saturation adjustment calculations into the color correction process, thereby introducing W-pixel information to achieve a better signal-to-noise ratio. Furthermore, compared to existing color correction algorithms that only calibrate one set of matrices under a single light source, this embodiment calibrates multiple color mapping matrices with different saturations, ensuring more accurate color correction.
[0072] The color restoration method based on RGBW images provided in this embodiment, taking into account the characteristics of RGBW image sensors, replaces the traditional statistical G / R and G / B white balance with the calculation of the error between the estimated W value of RGB and the true W value, thereby obtaining a more accurate AWB result; furthermore, by introducing the saturation adjustment function into the CCM process and using the extended CCM matrix with W participation, a color correction result with better signal-to-noise ratio and more accurate color is obtained.
[0073] The color restoration method based on RGBW images provided in this embodiment does not require the W pixels to be merged into the RGB pixels during the demosaicing stage. Instead, it places the positions where the W pixels are used in the CCM (Color Matrix Censor), thereby avoiding the destruction of color information accuracy during the demosaicing stage. Furthermore, this method uses the information from the W pixels to find gray pixels, achieving more accurate and stable automatic white balance processing for the image.
[0074] The color restoration method based on RGBW images provided in this embodiment calculates the probability of a pixel being gray by calibrating the mapping relationship between multiple sets of RGB and W response curves and calculating the error between the estimated W channel pixels and the actual W pixels. Then, it judges extreme color temperature light source scenes by summing the probabilities of all pixels being gray pixels, and performs white balance correction by calculating the mean of gray pixels. An extended color correction matrix is obtained by solving the four-channel image color data and changing the saturation of the target color. The extended color correction matrix yields an RGB image with better signal-to-noise ratio and more accurate colors.
[0075] This embodiment also provides a color restoration device based on RGBW images, used to implement the color restoration method based on RGBW images as described above. As shown in Figure 2, the color restoration device based on RGBW images includes: an offline acquisition module for acquiring the mapping relationship matrix between the RGB response curve and the W response curve and a four-channel color correction matrix at different saturations; an online acquisition module for acquiring the original RGBW image and acquiring the full-resolution R image, full-resolution G image, full-resolution B image, and full-resolution W image of the original RGBW image; an estimation calculation module for calculating the estimated value of W pixels using the mapping relationship matrix and the full-resolution R image, full-resolution G image, and full-resolution B image; a probability calculation module for calculating the probability that all pixels in the original RGBW image are gray pixels based on the full-resolution W image and the estimated value of W pixels; a white balance correction module for automatically white-balancing the RGB channels using the probability that all pixels in the original RGBW image are gray pixels to obtain an automatically white-balanced RGBW image; and a color correction module for color correction of the automatically white-balanced RGBW image using the four-channel color correction matrix to obtain a color-restored RGB image.
[0076] The color restoration device based on RGBW images provided in this embodiment obtains the mapping relationship between the RGB channels and the W channels and calculates the estimated value of the W pixels to obtain the probability of gray pixels. Then, it uses this probability to perform automatic white balance correction, which can complete more accurate and stable automatic white balance processing of the image. By performing color correction processing through a four-channel color correction matrix, the color correction process can use the W pixel information to obtain a better signal-to-noise ratio and more accurate color, thus solving the problem of how to perform color restoration on RGBW images to obtain RGB images with high signal-to-noise ratio and accurate color.
[0077] Furthermore, this embodiment also provides an electronic device, including a memory, a processor, and an executable program stored in the memory and capable of being run by the processor; when the processor runs the executable program, it performs the color restoration method based on RGBW images as described above.
[0078] Furthermore, this embodiment also provides a computer storage medium storing an executable program; when the executable program is executed, it implements the color restoration method based on RGBW images as described above.
[0079] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to mutually. In addition, different parts between embodiments can also be combined with each other, and this invention does not limit this.
[0080] The color restoration method, apparatus, electronic device, and computer storage medium based on RGBW images provided in this embodiment include: obtaining a mapping matrix between the RGB response curve and the W response curve, and a four-channel color correction matrix at different saturations; inputting an original RGBW image and obtaining a full-resolution R image, a full-resolution G image, a full-resolution B image, and a full-resolution W image of the original RGBW image; calculating W pixel estimates using the mapping matrix and the full-resolution R, G, and B images; calculating the probability that all pixels in the original RGBW image are gray pixels based on the full-resolution W image and the W pixel estimates; performing automatic white balance correction on the RGB channels using the probability that all pixels in the original RGBW image are gray pixels to obtain an automatically white balance corrected RGBW image; and performing color correction on the automatically white balance corrected RGBW image using the four-channel color correction matrix to obtain a color-restored RGB image. By obtaining the mapping relationship between the RGB channels and the W channels and calculating the estimated value of the W pixels to obtain the probability of gray pixels, and then using this probability to perform automatic white balance correction, more accurate and stable automatic white balance processing of the image can be achieved. By performing color correction processing through a four-channel color correction matrix, the color correction process can utilize the W pixel information to obtain a better signal-to-noise ratio and more accurate color, solving the problem of how to perform color restoration on RGBW images to obtain RGB images with high signal-to-noise ratio and accurate color.
[0081] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the present invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the claims.
Claims
1. A color restoration method based on RGBW images, characterized in that, include: Obtain the mapping matrix between the RGB response curve and the W response curve, and the four-channel color correction matrix under different saturations; Input the original RGBW image and obtain its full-resolution R, G, B, and W images; use the mapping matrix and the full-resolution R, G, and B images to calculate the estimated W pixel value. Based on the full-resolution W image and the estimated W pixel value, the probability that all pixels in the original RGBW image are gray pixels is calculated; using the probability that all pixels in the original RGBW image are gray pixels, automatic white balance correction is performed on the RGB channels to obtain the automatically white balance corrected RGBW image; using the four-channel color correction matrix, color correction is performed on the automatically white balance corrected RGBW image to obtain the color-restored RGB image.
2. The color restoration method based on RGBW images according to claim 1, characterized in that, The method for obtaining the mapping relationship matrix between the RGB response curve and the W response curve includes: obtaining multiple sets of pixel arrays based on color charts photographed under light sources of different color temperatures, each set of pixel arrays including R pixel values, G pixel values, B pixel values and W pixel values; linearly fitting the R pixel values, G pixel values, B pixel values and W pixel values to obtain the mapping relationship between the RGB response curve and the W response curve; and minimizing the mapping relationship to obtain the mapping relationship matrix between the RGB response curve and the W response curve.
3. The color restoration method based on RGBW images according to claim 1, characterized in that, The method for obtaining the four-channel color correction matrix under different saturation levels includes: obtaining the RGB color matrix of the color chart; using the RGB color matrix, calculating the linear standard RGB matrix with respect to saturation; photographing the color chart to obtain the RGBW color matrix; and performing a minimization objective solution on the linear standard RGB matrix and the RGBW color matrix to obtain the four-channel color correction matrix under different saturation levels.
4. The color restoration method based on RGBW images according to claim 1, characterized in that, The method for calculating the probability that all pixels in the original RGBW image are gray pixels based on the full-resolution W image and the estimated W pixel value includes: calculating the error value between the W pixel value of the current pixel in the full-resolution W image and the estimated W pixel value; calculating the probability that the current pixel is a gray pixel based on the error value; and traversing the full-resolution W image to obtain the probability that all pixels are gray pixels.
5. The color restoration method based on RGBW images according to claim 1, characterized in that, The method for automatic white balance correction of RGB channels using the probability that all pixels in the original RGBW image are gray pixels includes: determining whether the original RGBW image was taken under extreme color temperature light sources; if the original RGBW image was taken under extreme color temperature light sources, then no automatic white balance correction is performed, or an existing automatic white balance correction method is used; if the original RGBW image was not taken under extreme color temperature light sources, then the probability that all pixels in the original RGBW image are gray pixels is used to calculate the R channel gain value, G channel gain value, and B channel gain value, and the full-resolution R image is multiplied by the R channel gain value, the full-resolution G image is multiplied by the G channel gain value, and the full-resolution B image is multiplied by the B channel gain value to obtain the RGB channel values after automatic white balance correction.
6. The color restoration method based on RGBW images according to claim 5, characterized in that, The method for determining whether an original RGBW image was taken under extreme color temperature light sources includes: calculating the probability sum of pixels that are gray pixels in the original RGBW image; if the probability sum is less than the probability threshold, then the original RGBW image is determined to have been taken under extreme color temperature light sources, otherwise it is not taken under extreme color temperature light sources.
7. The color restoration method based on RGBW images according to claim 1, characterized in that, The method for color correction of an RGBW image after automatic white balance correction using a four-channel color correction matrix to obtain a color-restored RGB image includes: interpolating the four-channel color correction matrix according to the desired saturation to obtain a target color correction matrix; and using the target color correction matrix to color correct the RGBW image after automatic white balance correction to obtain a color-restored RGB image.
8. A color restoration apparatus based on an RGBW image, used to implement the color restoration method based on an RGBW image as described in any one of claims 1 to 7, characterized in that, The color restoration device based on RGBW images includes: an offline acquisition module for acquiring the mapping relationship matrix between the RGB response curve and the W response curve, and a four-channel color correction matrix at different saturations; an online acquisition module for acquiring the original RGBW image, and acquiring the full-resolution R image, full-resolution G image, full-resolution B image, and full-resolution W image of the original RGBW image; an estimation calculation module for calculating the estimated value of W pixels using the mapping relationship matrix and the full-resolution R image, full-resolution G image, and full-resolution B image; a probability calculation module for calculating the probability that all pixels in the original RGBW image are gray pixels based on the full-resolution W image and the estimated value of W pixels; a white balance correction module for automatically white-balancing the RGB channels using the probability that all pixels in the original RGBW image are gray pixels, to obtain an automatically white-balanced RGBW image; and a color correction module for color correction of the automatically white-balanced RGBW image using the four-channel color correction matrix, to obtain a color-restored RGB image.
9. An electronic device, characterized in that, It includes a memory, a processor, and an executable program stored in the memory and capable of being run by the processor; when the processor runs the executable program, it performs the color restoration method based on an RGBW image as described in any one of claims 1 to 7.
10. A computer storage medium, characterized in that, The computer storage medium stores an executable program; when the executable program is executed, it implements the color restoration method based on RGBW images as described in any one of claims 1 to 7.