Methods for removing mosaics, computer storage media, and electronic devices

By introducing a guided filtering method into Bayer format images for G-component interpolation and residual calculation, the problem of false color in optical imaging sensors is solved, and the quality of color images is improved.

CN115984109BActive Publication Date: 2026-11-17AMLOGIC (SHANGHAI) CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202211695180.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-12-28
Publication Date
2026-11-17
Estimated Expiration
2042-12-28

AI Technical Summary

Technical Problem

The color filter array in existing optical imaging sensors causes high-frequency information in Bayer images to be inaccurately reconstructed, resulting in problems such as false color and discontinuities.

Method used

A guided filtering method is used to interpolate the G components of non-G pixels in Bayer format images to restore color information. The G channel image is obtained by calculating the color residual, and the R and B channel images are reconstructed to improve the false color problem.

Benefits of technology

It effectively reduces unnatural colors in color images and improves image quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115984109B_ABST
    Figure CN115984109B_ABST
Patent Text Reader

Abstract

This invention discloses a method for de-mosaicing, a computer storage medium, and an electronic device. The de-mosaicing method includes: performing G-component interpolation on non-G pixels in a Bayer format image to obtain a first interpolated image of non-G pixels and a first interpolated image of G components in the horizontal direction, and obtaining a second interpolated image of non-G pixels and a second interpolated image of G components in the vertical direction; obtaining a first filtered image of G components in the horizontal direction using a guided filtering method based on the first interpolated image of non-G pixels and the first interpolated image of G components; and obtaining a second filtered image of G components in the vertical direction using a guided filtering method based on the second interpolated image of non-G pixels and the second interpolated image of G components. This method can improve the false color problem in the color image after mosaic removal and improve image quality.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for de-mosaicing, a computer storage medium, and an electronic device. Background Technology

[0002] Currently, mainstream optical imaging sensors use color filter arrays to sample input light, with Bayer filters being the most common. In the sampled Bayer image, RGB values ​​are arranged alternately in a regular pattern, with each pixel having only one color value. The number of G pixels is twice that of R or B pixels. Related technologies employ demosaic algorithms to reconstruct the RGB primary colors from each pixel in the Bayer image. However, because the data acquired by the sensor is limited by the sampling frequency of the filter, it cannot accurately reconstruct high-frequency information in the Bayer image, resulting in numerous false colors and discontinuities. Summary of the Invention

[0003] The present invention aims to at least solve one of the technical problems existing in the prior art. Therefore, one object of the present invention is to provide a method for removing mosaic artifacts, which can improve the problem of false color in color images after mosaic removal and improve image quality.

[0004] The second objective of this invention is to provide a computer storage medium.

[0005] The third objective of this invention is to provide an electronic device.

[0006] To address the aforementioned problems, a first aspect of the present invention provides a method for de-mosaicing, comprising: acquiring a Bayer format image, wherein the Bayer format image includes G pixels and non-G pixels, and the non-G pixels include R pixels and B pixels; performing G component interpolation on the non-G pixels in the Bayer format image to obtain a first non-G pixel interpolated image and a first G component interpolated image in the horizontal direction, and obtaining a second non-G pixel interpolated image and a second G component interpolated image in the vertical direction; and obtaining a first G component filtered image in the horizontal direction based on the first non-G pixel interpolated image and the first G component interpolated image using a guided filtering method. Furthermore, a second G-component filtered image in the vertical direction is obtained by guided filtering based on the second non-G pixel interpolated image and the second G-component interpolated image; a first G-component residual value is obtained based on the first G-component filtered image and the Bayer format image, and a second G-component residual value is obtained based on the second G-component filtered image and the Bayer format image; a G-channel image is obtained based on the first G-component residual value, the second G-component residual value, and the G-pixel image in the Bayer format image; an R-channel image and a B-channel image are reconstructed; and a color image after removing mosaic is obtained based on the G-channel image, the R-channel image, and the B-channel image.

[0007] According to the present invention, the method for removing mosaic artifacts involves interpolating the non-G pixels in a Bayer format image using the G component, and then introducing a guided filtering method to obtain a first G component filtered image in the horizontal direction and a second G component filtered image in the vertical direction. This effectively restores the color information in the Bayer image and reduces the appearance of unnatural colors in the image. Furthermore, color residual calculation is performed on this basis, and the G channel image is obtained using the first G component residual value, the second G component residual value, and the G pixel image in the Bayer format image. This effectively improves the false color problem in the color image after mosaic removal and enhances the quality of the color image.

[0008] In some embodiments, the G pixel includes Gr pixels horizontally adjacent to the R pixel and Gb pixels horizontally adjacent to the B pixel. The first non-G pixel interpolated image includes a first R pixel interpolated image and a first B pixel interpolated image. The first G component interpolated image includes a first Gr component interpolated image and a first Gb component interpolated image. The second non-G pixel interpolated image includes a second R pixel interpolated image and a second B pixel interpolated image. The second G component interpolated image includes a second Gr component interpolated image and a second Gb component interpolated image.

[0009] Performing G-component interpolation on non-G pixels in the Bayer format image to obtain a first non-G pixel interpolated image and a first G-component interpolated image in the horizontal direction, and obtaining a second non-G pixel interpolated image and a second G-component interpolated image in the vertical direction, includes:

[0010] The R pixel image and Gr pixel image in the Bayer format image are convolved in the horizontal direction to obtain the first Gr component convolved image of the R pixel and the first R component convolved image of the Gr pixel.

[0011] The first Gr component convolutional image is combined with the R pixel image to obtain the first Gr component interpolated image, and the first R component convolutional image is combined with the Gr pixel image to obtain the first R pixel interpolated image.

[0012] The B-pixel image and Gb-pixel image in the Bayer format image are convolved in the horizontal direction to obtain the first Gb component convolved image of the B-pixel and the first B component convolved image of the Gb-pixel.

[0013] The first Gb component convolutional image is combined with the B pixel image to obtain the first Gb component interpolated image, and the first B component convolutional image is combined with the Gb pixel image to obtain the first B pixel interpolated image.

[0014] The R-pixel image and Gb-pixel image in the Bayer format image are convolved in the vertical direction to obtain the second Gb component convolved image of the R-pixel and the second R component convolved image of the Gb-pixel.

[0015] The second Gb component convolution image is combined with the R pixel image to obtain the second Gb component interpolated image, and the second R component convolution image is combined with the Gb pixel image to obtain the second R pixel interpolated image.

[0016] The B-pixel image and Gr-pixel image in the Bayer format image are convolved in the vertical direction to obtain the second Gr component convolved image of the B-pixel and the second B component convolved image of the Gr-pixel.

[0017] The second Gr component convolutional image is combined with the B pixel image to obtain the second Gr component interpolated image, and the second B component convolutional image is combined with the Gr pixel image to obtain the second B pixel interpolated image.

[0018] In some embodiments, the first G component filtered image includes a first Gr component filtered image and a first Gb component filtered image. Obtaining the first G component filtered image in the horizontal direction using a guided filtering method based on the first non-G pixel interpolated image and the first G component interpolated image includes: using the first R pixel interpolated image as a guide image, and filtering the first Gr component interpolated image using the guided filtering method to obtain the first Gr component filtered image; using the first B pixel interpolated image as a guide image, and filtering the first Gb component interpolated image using the guided filtering method to obtain the first Gb component filtered image.

[0019] In some embodiments, the second G component filtered image includes a second Gr component filtered image and a second Gb component filtered image. Obtaining the second G component filtered image in the vertical direction using a guided filtering method based on the second non-G pixel interpolated image and the second G component interpolated image includes: using the second R pixel interpolated image as a guide image, and filtering the second Gb component interpolated image using the guided filtering method to obtain the second Gb component filtered image; using the second B pixel interpolated image as a guide image, and filtering the second Gr component interpolated image using the guided filtering method to obtain the second Gr component filtered image.

[0020] In some embodiments, the first G component residual value includes a first Gr component residual value and a first Gb component residual value, and the second G component residual value includes a second Gr component residual value and a second Gb component residual value. Obtaining the first G component residual value based on the first G component filtered image and the Bayer format image, and obtaining the second G component residual value based on the second G component filtered image and the Bayer format image, includes: performing difference processing on the first Gr component filtered image and the Bayer format image to obtain the first Gr component residual value; performing difference processing on the first Gb component filtered image and the Bayer format image to obtain the first Gb component residual value; performing difference processing on the second Gr component filtered image and the Bayer format image to obtain the second Gr component residual value; and performing difference processing on the second Gb component filtered image and the Bayer format image to obtain the second Gb component residual value.

[0021] In some embodiments, obtaining a G-channel image based on a first G-component residual value, a second G-component residual value, and a G-pixel image in the Bayer format image includes: performing low-pass filtering on the first G-component residual value and the second G-component residual value respectively; obtaining a first G-component corrected image based on the low-pass filtered first G-component residual value and the first G-component interpolated image; obtaining a second G-component corrected image based on the low-pass filtered second G-component residual value and the second G-component interpolated image; and obtaining the G-channel image based on the G-pixel image in the Bayer format image, the first G-component corrected image, and the second G-component corrected image.

[0022] In some embodiments, the first G-component corrected image includes a first Gr-component corrected image and a first Gb-component corrected image, and the second G-component corrected image includes a second Gr-component corrected image and a second Gb-component corrected image. Obtaining the G-channel image based on the G-pixel image in the Bayer format image, the first G-component corrected image, and the second G-component corrected image includes: combining the G-pixel image, the first Gr-component corrected image, and the first Gb-component corrected image to obtain a first G-component image in the horizontal direction; combining the G-pixel image, the second Gr-component corrected image, and the second Gb-component corrected image to obtain a second G-component image in the vertical direction; calculating a first gradient value in the horizontal direction and a second gradient value in the vertical direction for each pixel in the Bayer format image; and performing a weighted average of the first G-component image and the second G-component image based on the first gradient value and the second gradient value to obtain the G-channel image.

[0023] In some embodiments, reconstructing the R-channel image and the B-channel image includes: calculating the third gradient value of the R component of each pixel in the Bayer format image under the interpolation direction and the fourth gradient value of the B component of each pixel under the interpolation direction, wherein the interpolation direction includes the horizontal direction, the vertical direction, the 45-degree direction, and the 135-degree direction; interpolating the R component according to the third gradient value to reconstruct the R-channel image; and interpolating the B component according to the fourth gradient value to reconstruct the B-channel image.

[0024] A second aspect of the present invention provides a computer storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the de-mosaic method described in the above embodiments.

[0025] A third aspect of the present invention provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the at least one processor executes the computer program to implement the de-mosaic method described in the above embodiments.

[0026] According to the electronic device of the present invention, by executing the de-mosaic method provided in the above embodiments through a processor, the problem of false color in the color image after mosaic removal can be improved, thereby improving image quality.

[0027] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0028] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0029] Figure 1 This is a flowchart of a method for removing mosaic according to an embodiment of the present invention;

[0030] Figure 2 This is a schematic diagram of a Bayer format image according to an embodiment of the present invention;

[0031] Figure 3 This is a structural block diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0032] The embodiments of the present invention are described in detail below. The embodiments described with reference to the accompanying drawings are exemplary. The embodiments of the present invention are described in detail below.

[0033] To address the aforementioned problems, a first aspect of this invention proposes a method for removing mosaic artifacts. This method can improve the false color problem in color images after mosaic removal and enhance image quality.

[0034] The following is for reference. Figure 1 The method for de-mosaicing according to embodiments of the present invention is described as follows: Figure 1 As shown, the method includes at least steps S1-S7.

[0035] Step S1: Obtain a Bayer format image, wherein the Bayer format image includes G pixels and non-G pixels, and the non-G pixels include R pixels and B pixels. For example, ... Figure 2 The image shown is a Bayer format image with a 5×5 distribution, where R, B, and G pixels are arranged alternately according to a pattern, and each pixel has only one color value. Based on this, the RGB three primary colors are reconstructed at each pixel in the Bayer format image using a demosaic algorithm. This involves first subtracting the G component at the R and B pixels, and then reconstructing the R and B components. The specific steps are as follows.

[0036] Step S2: Perform G component interpolation on the non-G pixels in the Bayer format image to obtain a first non-G pixel interpolated image and a first G component interpolated image in the horizontal direction, and obtain a second non-G pixel interpolated image and a second G component interpolated image in the vertical direction.

[0037] Step S3 involves obtaining a first G-component filtered image in the horizontal direction using a guided filtering method based on the first non-G pixel interpolated image and the first G-component interpolated image, and obtaining a second G-component filtered image in the vertical direction using a guided filtering method based on the second non-G pixel interpolated image and the second G-component interpolated image. That is, after interpolating the non-G pixels using the G-component, the color residual is not directly calculated. Instead, the first and second G-component interpolated images are first filtered using a guided filtering method to recover the color information in the Bayer image, achieving a smoothing effect for the G-component interpolated image. Then, the color residual is calculated based on this. Therefore, when obtaining the G-channel image using the G-component residual value calculated from the guided-filtered G-component interpolated image, the false color problem in the color image after mosaic removal can be effectively improved, thus enhancing the color image quality.

[0038] Step S4: Obtain the first G component residual value based on the first G component filtered image and the Bayer format image, and obtain the second G component residual value based on the second G component filtered image and the Bayer format image.

[0039] Step S5: Obtain the G-channel image based on the first G-component residual value, the second G-component residual value, and the G-pixel image in the Bayer format image.

[0040] Step S6: Reconstruct the R channel image and the B channel image.

[0041] Step S7: Obtain the color image after removing the mosaic from the G channel image, R channel image, and B channel image.

[0042] According to the present invention, the method for removing mosaic artifacts involves interpolating the non-G pixels in a Bayer format image using the G component, and then introducing a guided filtering method to obtain a first G component filtered image in the horizontal direction and a second G component filtered image in the vertical direction. This effectively restores the color information in the Bayer image and reduces the appearance of unnatural colors in the image. Furthermore, color residual calculation is performed on this basis, and the G channel image is obtained using the first G component residual value, the second G component residual value, and the G pixel image in the Bayer format image. This effectively improves the false color problem in the color image after mosaic removal and enhances the quality of the color image.

[0043] In some embodiments, G pixels include Gr pixels that are horizontally adjacent to R pixels and Gb pixels that are horizontally adjacent to B pixels. The first non-G pixel interpolated image includes a first R pixel interpolated image and a first B pixel interpolated image. The first G component interpolated image includes a first Gr component interpolated image and a first Gb component interpolated image. The second non-G pixel interpolated image includes a second R pixel interpolated image and a second B pixel interpolated image. The second G component interpolated image includes a second Gr component interpolated image and a second Gb component interpolated image.

[0044] Based on the above description, this application obtains a first non-G pixel interpolated image and a first G component interpolated image in the horizontal direction, and a second non-G pixel interpolated image and a second G component interpolated image in the vertical direction through the following methods.

[0045] The R pixel image and Gr pixel image in the Bayer format image are convolved in the horizontal direction to obtain the first Gr component convolved image and the first R component convolved image of the R pixel; the first Gr component convolved image is combined with the R pixel image to obtain the first Gr component interpolated image, and the first R component convolved image is combined with the Gr pixel image to obtain the first R pixel interpolated image.

[0046] For example, the R pixel image and Gr pixel image in the acquired Bayer format image are subjected to circular convolution processing in the horizontal direction using a general template such as [1 / 2, 0, 1 / 2] to obtain the first Gr component convolution image and the first R component convolution image of the R pixel. Then, the first Gr component convolution image and the first R component convolution image are combined with the R pixel image and the Gr pixel image respectively to obtain the first Gr component interpolated image and the first R pixel interpolated image in the horizontal direction.

[0047] The B-pixel image and Gb-pixel image in the Bayer format image are convolved in the horizontal direction to obtain the first Gb component convolved image of the B-pixel and the first B component convolved image of the Gb-pixel; the first Gb component convolved image is combined with the B-pixel image to obtain the first Gb component interpolated image, and the first B component convolved image is combined with the Gb-pixel image to obtain the first B-pixel interpolated image.

[0048] For example, the B-pixel image and Gb-pixel image in the acquired Bayer format image are subjected to a general template such as [1 / 2, 0, 1 / 2] and a circular convolution process is performed in the horizontal direction to obtain the first Gb component convolution image of the B-pixel and the first B component convolution image of the Gb-pixel. Then, the first Gb component convolution image and the first B component convolution image are combined with the B-pixel image and the Gb-pixel image respectively to obtain the first Gb component interpolated image and the first B-pixel interpolated image in the horizontal direction.

[0049] The R-pixel image and Gb-pixel image in the Bayer format are convolved in the vertical direction to obtain the second Gb component convolved image of the R-pixel and the second R component convolved image of the Gb-pixel. The second Gb component convolved image is combined with the R-pixel image to obtain the second Gb component interpolated image, and the second R component convolved image is combined with the Gb-pixel image to obtain the second R-pixel interpolated image.

[0050] For example, the R-pixel image and Gb-pixel image in the acquired Bayer format image are subjected to cyclic convolution in the vertical direction using a general template such as [1 / 2, 0, 1 / 2] to obtain the second Gb component convolution image of the R-pixel and the second R component convolution image of the Gb-pixel. Then, the second Gb component convolution image and the second R component convolution image are combined with the R-pixel image and the Gb-pixel image respectively to obtain the second Gb component interpolated image and the second R-pixel interpolated image in the vertical direction.

[0051] The B-pixel image and Gr-pixel image in the Bayer format are convolved in the vertical direction to obtain the second Gr component convolved image of the B-pixel and the second B component convolved image of the Gr-pixel. The second Gr component convolved image is combined with the B-pixel image to obtain the second Gr component interpolated image, and the second B component convolved image is combined with the Gr-pixel image to obtain the second B-pixel interpolated image.

[0052] For example, the B-pixel image and Gr-pixel image in the acquired Bayer format image are subjected to a general template such as [1 / 2, 0, 1 / 2], and a circular convolution process is performed in the vertical direction to obtain the second Gr component convolution image of the B-pixel and the second B component convolution image of the Gr-pixel. Then, the second Gr component convolution image and the second B component convolution image are combined with the B-pixel image and the Gr-pixel image respectively to obtain the second Gr component interpolated image and the second B-pixel interpolated image in the vertical direction.

[0053] In some embodiments, the first G component filtered image includes a first Gr component filtered image and a first Gb component filtered image. Based on this, this application uses the first R pixel interpolation image as a guide map and performs guided filtering on the first Gr component interpolation image to obtain the first Gr component filtered image; it also uses the first B pixel interpolation image as a guide map and performs guided filtering on the first Gb component interpolation image to obtain the first Gb component filtered image. Thus, by filtering the first G component interpolation image using guided filtering, the color information in the Bayer image is recovered, achieving a smoothing effect on the G component interpolation image. Furthermore, the color residual is calculated based on this. Therefore, when the G channel image is subsequently obtained using the G component residual value calculated from the guided-filtered G component interpolation image, the false color problem in the color image after mosaic removal can be effectively improved, thus enhancing the color image quality.

[0054] In some embodiments, the second G component filtered image includes a second Gr component filtered image and a second Gb component filtered image. Based on this, this application uses the second R pixel interpolation image as a guide map and performs guided filtering on the second Gb component interpolation image to obtain the second Gb component filtered image; similarly, it uses the second B pixel interpolation image as a guide map and performs guided filtering on the second Gr component interpolation image to obtain the second Gr component filtered image. Thus, by filtering the second G component interpolation image using guided filtering, the color information in the Bayer image is recovered, achieving a smoothing effect on the G component interpolation image. Furthermore, the color residual is calculated based on this. Therefore, when the G channel image is subsequently obtained using the G component residual value calculated from the guided-filtered G component interpolation image, the false color problem in the color image after mosaic removal can be effectively improved, thus enhancing the color image quality.

[0055] In some embodiments, the first G component residual value includes a first Gr component residual value and a first Gb component residual value, and the second G component residual value includes a second Gr component residual value and a second Gb component residual value. Based on this, this application obtains the first Gr component residual value by performing difference processing on the first Gr component filtered image and the Bayer format image; performs difference processing on the first Gb component filtered image and the Bayer format image; performs difference processing on the second Gr component filtered image and the Bayer format image; and performs difference processing on the second Gb component filtered image and the Bayer format image to obtain the second Gb component residual value.

[0056] For example, the first Gr component filtered image is subtracted point-to-point from the Bayer format image, and only the difference corresponding to the Gr pixels in the Bayer format image is retained as the first Gr component residual value; the first Gb component filtered image is subtracted point-to-point from the Bayer format image, and only the difference corresponding to the Gb pixels in the Bayer format image is retained as the first Gb component residual value; the second Gr component filtered image is subtracted point-to-point from the Bayer format image, and only the difference corresponding to the Gr pixels in the Bayer format image is retained as the second Gr component residual value; the second Gb component filtered image is subtracted point-to-point from the Bayer format image, and only the difference corresponding to the Gb pixels in the Bayer format image is retained as the second Gb component residual value.

[0057] In some embodiments, after obtaining the first G component residual value and the second G component residual value, this application first performs low-pass filtering on the first G component residual value and the second G component residual value respectively; obtains a first G component corrected image based on the first G component residual value after low-pass filtering and the first G component interpolated image; obtains a second G component corrected image based on the second G component residual value after low-pass filtering and the second G component interpolated image; and obtains a G channel image based on the G pixel image in the Bayer format image, the first G component corrected image, and the second G component corrected image.

[0058] For example, a low-pass smoothing operator such as [1 / 2, 0, 1 / 2] is convolved with the first Gr component residual value to obtain a smoothed first Gr component residual value; and a low-pass smoothing operator such as [1 / 2, 0, 1 / 2] is convolved with the first Gb component residual value to obtain a smoothed first Gb component residual value; and a low-pass smoothing operator such as [1 / 2, 0, 1 / 2] is convolved with the second Gr component residual value to obtain a smoothed second Gr component residual value; and a low-pass smoothing operator such as [1 / 2, 0, 1 / 2] is convolved with the second Gb component residual value to obtain a smoothed second Gb component residual value.

[0059] Then, the smoothed residual value of the first Gr component is added to the interpolated image of the first Gr component to obtain the corrected image of the first Gr component; and the smoothed residual value of the second Gb component is added to the interpolated image of the second Gb component to obtain the corrected image of the second Gb component; the smoothed residual value of the second Gr component is added to the interpolated image of the second Gr component to obtain the corrected image of the second Gr component; and the smoothed residual value of the first Gb component is added to the interpolated image of the first Gb component to obtain the corrected image of the first Gb component.

[0060] In some embodiments, the first G component corrected image includes a first Gr component corrected image and a first Gb component corrected image, and the second G component corrected image includes a second Gr component corrected image and a second Gb component corrected image. Based on this, this application combines the G pixel image, the first Gr component corrected image, and the first Gb component corrected image to obtain the first G component image in the horizontal direction; combines the G pixel image, the second Gr component corrected image, and the second Gb component corrected image to obtain the second G component image in the vertical direction; calculates the first gradient value in the horizontal direction and the second gradient value in the vertical direction for each pixel in the Bayer format image; and performs a weighted average of the first G component image and the second G component image based on the first gradient value and the second gradient value to obtain the G channel image.

[0061] For example, the G-channel image = (first G component image * second gradient value + second G component image * first gradient value) / (first gradient value + second gradient value).

[0062] In some embodiments, this application calculates the third gradient value of the R component of each pixel in the Bayer format image under the interpolation direction and the fourth gradient value of the B component of each pixel under the interpolation direction, wherein the interpolation direction includes the horizontal direction, the vertical direction, the 45-degree direction, and the 135-degree direction; interpolates the R component according to the third gradient value to reconstruct the R channel image; and interpolates the B component according to the fourth gradient value to reconstruct the B channel image.

[0063] A second aspect of the present invention provides a computer storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the de-mosaic method provided in the above embodiments.

[0064] A third aspect of the present invention provides an electronic device, such as... Figure 3 As shown, the electronic device 10 includes at least one processor 1 and a memory 2 communicatively connected to at least one processor 1.

[0065] The memory 2 stores a computer program that can be executed by at least one processor 1. When the at least one processor 1 executes the computer program, it implements the de-mosaic method provided in the above embodiments.

[0066] It should be noted that the specific implementation of the electronic device 10 in this embodiment of the invention is similar to the specific implementation of the de-mosaic method in any of the above embodiments of the invention. For details, please refer to the description of the method section. To reduce redundancy, it will not be repeated here.

[0067] According to the present invention, the electronic device 10, by executing the de-mosaic method provided in the above embodiment through the processor 1, can improve the false color problem in the color image after de-mosaic and improve the image quality.

[0068] In the description of this specification, any process or method described in the flowcharts or otherwise herein may be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order according to the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0069] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0070] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any of the following techniques known in the art, or a combination thereof: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0071] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0072] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0073] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

[0074] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example.

[0075] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method of demosaicking, characterized by, include: Obtain a Bayer format image, wherein the Bayer format image includes G pixels and non-G pixels, and the non-G pixels include R pixels and B pixels; The G-component interpolation of non-G pixels in the Bayer format image includes: performing convolution processing on the corresponding pixel images of the Bayer format image in the horizontal direction, and combining the convolutional image with the original pixel image to obtain a first non-G pixel interpolation image and a first G-component interpolation image in the horizontal direction; and performing convolution processing on the corresponding pixel images of the Bayer format image in the vertical direction, and combining the convolutional image with the original pixel image to obtain a second non-G pixel interpolation image and a second G-component interpolation image in the vertical direction. The first non-G pixel interpolated image is used as a guide image, and the first G component interpolated image is filtered by the guide filtering method to obtain the first G component filtered image in the horizontal direction. The second non-G pixel interpolated image is used as a guide image, and the second G component interpolated image is filtered by the guide filtering method to obtain the second G component filtered image in the vertical direction. The first G-component residual value is obtained based on the first G-component filtered image and the Bayer format image, and the second G-component residual value is obtained based on the second G-component filtered image and the Bayer format image; The G-channel image is obtained based on the first G-component residual value, the second G-component residual value, and the G-pixel image in the Bayer format image; Reconstruct the R-channel and B-channel images; A color image after removing mosaic is obtained based on the G channel image, the R channel image, and the B channel image.

2. The method of demosaicking according to claim 1, wherein, The G-pixel includes Gr-pixels horizontally adjacent to the R-pixel and Gb-pixels horizontally adjacent to the B-pixel. The first non-G-pixel interpolated image includes a first R-pixel interpolated image and a first B-pixel interpolated image. The first G-component interpolated image includes a first Gr-component interpolated image and a first Gb-component interpolated image. The second non-G-pixel interpolated image includes a second R-pixel interpolated image and a second B-pixel interpolated image. The second G-component interpolated image includes a second Gr-component interpolated image and a second Gb-component interpolated image. Performing G-component interpolation on non-G pixels in the Bayer format image to obtain a first non-G pixel interpolated image and a first G-component interpolated image in the horizontal direction, and obtaining a second non-G pixel interpolated image and a second G-component interpolated image in the vertical direction, includes: The R pixel image and Gr pixel image in the Bayer format image are convolved in the horizontal direction to obtain the first Gr component convolved image of the R pixel and the first R component convolved image of the Gr pixel. The first Gr component convolutional image is combined with the R pixel image to obtain the first Gr component interpolated image, and the first R component convolutional image is combined with the Gr pixel image to obtain the first R pixel interpolated image. The B-pixel image and Gb-pixel image in the Bayer format image are convolved in the horizontal direction to obtain the first Gb component convolved image of the B-pixel and the first B component convolved image of the Gb-pixel. The first Gb component convolutional image is combined with the B pixel image to obtain the first Gb component interpolated image, and the first B component convolutional image is combined with the Gb pixel image to obtain the first B pixel interpolated image. The R-pixel image and Gb-pixel image in the Bayer format image are convolved in the vertical direction to obtain the second Gb component convolved image of the R-pixel and the second R component convolved image of the Gb-pixel. The second Gb component convolution image is combined with the R pixel image to obtain the second Gb component interpolated image, and the second R component convolution image is combined with the Gb pixel image to obtain the second R pixel interpolated image. The B-pixel image and Gr-pixel image in the Bayer format image are convolved in the vertical direction to obtain the second Gr component convolved image of the B-pixel and the second B component convolved image of the Gr-pixel. The second Gr component convolutional image is combined with the B pixel image to obtain the second Gr component interpolated image, and the second B component convolutional image is combined with the Gr pixel image to obtain the second B pixel interpolated image.

3. The method of demosaicking according to claim 2, wherein, The first G-component filtered image includes a first Gr-component filtered image and a first Gb-component filtered image. The first G-component filtered image in the horizontal direction is obtained by guided filtering based on the first non-G pixel interpolation image and the first G-component interpolation image, including: The first R pixel interpolated image is used as a guide image, and the first Gr component interpolated image is filtered by the guide filtering method to obtain the first Gr component filtered image. The first B-pixel interpolated image is used as a guide image, and the first Gb component interpolated image is filtered by the guide filtering method to obtain the first Gb component filtered image.

4. The method of demosaicking according to claim 3, wherein, The second G component filtered image includes a second Gr component filtered image and a second Gb component filtered image. The second G component filtered image in the vertical direction is obtained by guided filtering based on the second non-G pixel interpolated image and the second G component interpolated image, including: The second R pixel interpolated image is used as a guide image, and the second Gb component interpolated image is filtered by the guide filtering method to obtain the second Gb component filtered image. The second B pixel interpolated image is used as a guide image, and the second Gr component interpolated image is filtered by the guide filtering method to obtain the second Gr component filtered image.

5. The method of demosaicking according to claim 4, wherein, The first G component residual value includes a first Gr component residual value and a first Gb component residual value, and the second G component residual value includes a second Gr component residual value and a second Gb component residual value. Obtaining the first G component residual value based on the first G component filtered image and the Bayer format image, and obtaining the second G component residual value based on the second G component filtered image and the Bayer format image, includes: The first Gr component filtered image is compared with the Bayer format image to obtain the residual value of the first Gr component. The first Gb component filtered image is compared with the Bayer format image to obtain the first Gb component residual value. The second Gr component filtered image is compared with the Bayer format image to obtain the second Gr component residual value; The second Gb component filtered image is compared with the Bayer format image to obtain the second Gb component residual value.

6. The method of demosaicking according to claim 5, wherein, Obtaining the G-channel image based on the first G-component residual value, the second G-component residual value, and the G-pixel image in the Bayer format image includes: The residual values ​​of the first G component and the residual values ​​of the second G component are respectively subjected to low-pass filtering. The first G component correction image is obtained based on the first G component residual value after low-pass filtering and the first G component interpolated image. The corrected image of the second G component is obtained based on the residual value of the second G component after low-pass filtering and the interpolated image of the second G component. The G-channel image is obtained from the G-pixel image, the first G-component corrected image, and the second G-component corrected image in the Bayer format image.

7. The method of demosaicking according to claim 6, wherein, The first G-component corrected image includes a first Gr-component corrected image and a first Gb-component corrected image, and the second G-component corrected image includes a second Gr-component corrected image and a second Gb-component corrected image. The G-channel image is obtained based on the G-pixel image in the Bayer format image, the first G-component corrected image, and the second G-component corrected image, including: The G-pixel image, the first Gr component correction image, and the first Gb component correction image are combined to obtain the first G component image in the horizontal direction. The G-pixel image, the second Gr component correction image, and the second Gb component correction image are combined to obtain the second G component image in the vertical direction; Calculate the first gradient value in the horizontal direction and the second gradient value in the vertical direction for each pixel in the Bayer format image; The first G-component image and the second G-component image are weighted and averaged according to the first gradient value and the second gradient value to obtain the G-channel image.

8. The method of demosaicking of claim 1, wherein, Reconstruct the R-channel and B-channel images, including: Calculate the third gradient value of the R component of each pixel in the Bayer format image under the interpolation direction and the fourth gradient value of the B component of each pixel under the interpolation direction, wherein the interpolation direction includes the horizontal direction, the vertical direction, the 45-degree direction and the 135-degree direction; The R component is interpolated based on the third gradient value to reconstruct the R channel image; interpolating the B component according to the fourth gradient value to reconstruct the B channel image.

9. A computer storage medium having stored thereon a computer program, characterized in that The computer program, when executed by a processor, implements the demosaicing method of any one of claims 1-8.

10. An electronic device, comprising: comprising: at least one processor; a memory connected with the at least one processor in communication; wherein the memory has stored thereon computer program executable by the at least one processor to implement the demosaicing method of any one of claims 1-8 when executed by the at least one processor.

Citation Information

Patent Citations

  • Fast bayer color reconstruction method based on multi-guide filtering

    CN109658358A

  • Bayer image joint demosaicing and denoising method based on guided filtering

    CN111539893A