Demosaicing method, system and device and computer readable storage medium
By acquiring the pixel gradient and color difference principles, simultaneous interpolation of R channels, G channels, and B channels is achieved, which solves the problem of low image quality in the prior art, and improves the recovery effect and calculation speed of color images.
Patent Information
- Application Number
- CN202310499792.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-04
- Publication Date
- 2024-11-08
AI Technical Summary
The existing demosaic algorithm has low image quality that interpolates R channel, G channel, and B channel simultaneously, making it difficult to effectively restore color images.
By obtaining the G-channel pixel gradient in the neighborhood of the target pixel point in the grayscale image, determining the interpolated point gradient and converting it into weights, and obtaining the pixel estimate value in combination with the chromatic difference principle, achieving simultaneous interpolation of R channels, G channels, and B channels.
The image quality of interpolation of R channel, G channel, and B channel is improved, and the image detail recovery capability and calculation speed are enhanced, which is suitable for real-time color recovery of ultra-high-definition images.
Smart Images

Figure CN118917999A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to a de-mosaicing method, system, device and computer-readable storage medium. Background Art
[0002] According to the arrangement of the color filter array, the correlation between the channels in the color and spectral domains is used for interpolation, the missing two channel values are estimated, and the process of restoring a full-color image is called demosaicing. As an important hub for images from Bayer format to RGB format, the demosaicing algorithm interpolates the grayscale image in the Bayer format into a color image, and the reconstruction quality of the color image will directly depend on the quality of the demosaicing algorithm. However, the demosaicing algorithm in the prior art simultaneously interpolates the R channel, G channel, and B channel to reconstruct the image with low quality.
[0003] Therefore, it is necessary to provide a new de-mosaicing method, system, device and computer-readable storage medium to solve the above-mentioned problems existing in the prior art. Summary of the invention
[0004] The object of the present invention is to provide a de-mosaicing method, system, device and computer-readable storage medium to improve the image quality of the R channel, G channel and B channel interpolation at the same time.
[0005] To achieve the above object, the de-mosaicing method of the present invention comprises:
[0006] Get the pixel gradient of each G channel pixel in the neighborhood of the target pixel in the grayscale image;
[0007] Determine an interpolation point gradient according to the pixel gradient;
[0008] Converting the interpolation point gradient into a weight;
[0009] Get the pixel estimation value of the target pixel;
[0010] An interpolation result is obtained according to the weight and the pixel estimation value.
[0011] The beneficial effect of the de-mosaicing method is that the pixel gradient of each G channel pixel point in the neighborhood of the target pixel point in the grayscale image is obtained, the interpolation point gradient is determined according to the pixel gradient, the interpolation point gradient is converted into a weight, the pixel estimation value of the target pixel point is obtained, and the interpolation result is obtained according to the weight and the pixel estimation value, thereby improving the image quality of the simultaneous interpolation of the R channel, the G channel, and the B channel.
[0012] Optionally, before executing the step of obtaining the pixel gradient of each G channel pixel in the neighborhood of the target pixel, the step further includes:
[0013] determining a pixel array of a grayscale image;
[0014] The color channels of the target pixel are determined according to the pixel array, and the color channels include an R channel, a G channel, and a B channel.
[0015] Optionally, before executing the step of obtaining the pixel gradient of each G channel pixel in the neighborhood of the target pixel, the step further includes:
[0016] Perform boundary extension on a grayscale image.
[0017] Optionally, the step of extending the boundary of the grayscale image includes:
[0018] Taking the boundary of the grayscale image as the axis of symmetry, a plurality of rows or columns of pixel points at the boundary of the grayscale image are mirrored.
[0019] Optionally, the pixel gradient includes a horizontal pixel gradient and a vertical pixel gradient, the interpolation point gradient includes a horizontal interpolation point gradient and a vertical interpolation point gradient, and determining the interpolation point gradient according to the pixel gradient includes:
[0020] Performing a convolution operation on all the horizontal pixel gradients and the interpolation parameter to obtain the horizontal interpolation point gradient;
[0021] All the vertical pixel gradients are convolved with the interpolation parameters to obtain the vertical interpolation point gradient. The beneficial effect is that the horizontal pixel gradient and vertical pixel gradient of the G channel in the neighborhood reflect the change gradient in the neighborhood, thereby improving the accuracy of the interpolation of the image detail area.
[0022] Optionally, the interpolation point gradient includes a horizontal interpolation point gradient and a vertical interpolation point gradient, and converting the interpolation point gradient into a weight includes:
[0023] The difference between the horizontal interpolation point gradient and the vertical interpolation point gradient is nonlinearly mapped to the interval [0, 1] through a logic function to obtain a weight.
[0024] Optionally, the logic function is Among them, w h represents the weight, v h represents the horizontal interpolation point gradient, v v The beneficial effect is that the conversion from gradient to weight is realized through the logic function, so that the interpolation result is more in line with the edge direction of the image details.
[0025] Optionally, obtaining the pixel estimation value of the target pixel point includes:
[0026] The pixel estimation value of the target pixel point is obtained by using the color difference principle, and the pixel estimation value includes a horizontal pixel estimation value and a vertical pixel estimation value.
[0027] Optionally, the obtaining of the pixel estimation value of the target pixel further includes:
[0028] The horizontal pixel estimation value and the vertical pixel estimation value are corrected by Laplace gradient correction term. The beneficial effect is that the accuracy of the interpolation result is improved.
[0029] The present invention also provides a system for implementing the de-mosaicing method, the system comprising a pixel gradient acquisition unit, an interpolation point gradient determination unit, a weight conversion unit, a pixel estimation value acquisition unit and an interpolation result calculation unit, the pixel gradient acquisition unit being used to acquire the pixel gradient of each G channel pixel point in the neighborhood of the target pixel point in the grayscale image; the interpolation point gradient determination unit being used to determine the interpolation point gradient according to the pixel gradient; the weight conversion unit being used to convert the interpolation point gradient into a weight; the pixel estimation value acquisition unit being used to acquire the pixel estimation value of the target pixel point; and the interpolation result calculation unit being used to obtain the interpolation result according to the weight and the pixel estimation value.
[0030] Optionally, the system also includes an arrangement determination unit, a color channel determination unit and a boundary extension unit, the arrangement determination unit is used to determine the arrangement of the CFA array of the grayscale image, the color channel determination unit is used to determine the color channel of the target pixel point according to the arrangement of the CFA array, the color channel includes an R channel, a G channel and a B channel, and the boundary extension unit is used to extend the boundary of the grayscale image.
[0031] The present invention also provides a device, comprising:
[0032] processor;
[0033] a memory for storing processor-executable instructions;
[0034] Wherein, when the processor is configured to execute instructions in the memory, the de-mosaicing method is implemented.
[0035] The present invention also provides a computer-readable storage medium, comprising computer instructions, and when the computer instructions are executed by a processor, the de-mosaicing method is implemented. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 A flowchart of a demosaicing method in some embodiments of the present invention;
[0037] Figure 2is a schematic diagram of a CFA array of a grayscale image in some embodiments of the present invention;
[0038] Figure 3 A schematic diagram of a CFA array after a grayscale image is subjected to a first-stage boundary expansion in some embodiments of the present invention;
[0039] Figure 4 A schematic diagram of a CFA array after a grayscale image is subjected to a second-stage boundary extension in some embodiments of the present invention;
[0040] Figure 5 A schematic diagram of horizontal interpolation filtering of missing green pixels at blue pixel sampling points in some embodiments of the present invention;
[0041] Figure 6 A schematic diagram of vertical interpolation filtering of missing green pixels at blue pixel sampling points in some embodiments of the present invention;
[0042] Figure 7 A schematic diagram of a horizontal interpolation filter for missing red pixels at blue pixel sampling points in some embodiments of the present invention;
[0043] Figure 8 A schematic diagram of a vertical interpolation filter for missing red pixels at blue pixel sampling points in some embodiments of the present invention;
[0044] Fig. 9 A schematic diagram of a horizontal interpolation filter for missing green pixels at red pixel sampling points in some embodiments of the present invention;
[0045] Fig.10 A schematic diagram of a vertical interpolation filter for missing green pixels at red pixel sampling points in some embodiments of the present invention;
[0046] Fig.11 A schematic diagram of a horizontal interpolation filter for missing blue pixels at red pixel sampling points in some embodiments of the present invention;
[0047] Fig.12 A schematic diagram of a vertical interpolation filter for missing blue pixels at red pixel sampling points in some embodiments of the present invention;
[0048] Fig.13 A schematic diagram of a horizontal interpolation filter with blue pixels missing at green pixel sampling points in a row where blue pixels are located in some embodiments of the present invention;
[0049] Fig.14 A schematic diagram of a vertical interpolation filter with blue pixel points missing at green pixel sampling points in a row where blue pixels are located in some embodiments of the present invention;
[0050] Fig.15 A schematic diagram of a horizontal interpolation filter in which red pixels are missing at green pixel sampling points in a row where blue pixels are located in some embodiments of the present invention;
[0051] Fig.16 A schematic diagram of a vertical interpolation filter with missing red pixels at green pixel sampling points in a row where blue pixels are located in some embodiments of the present invention;
[0052] Fig.17 A schematic diagram of a horizontal interpolation filter in which blue pixels are missing at green pixel sampling points in a row where red pixels are located in some embodiments of the present invention;
[0053] Fig.18 A schematic diagram of a vertical interpolation filter with blue pixels missing at green pixel sampling points in a row where red pixels are located in some embodiments of the present invention;
[0054] Fig.19 A schematic diagram of a horizontal interpolation filter with missing red pixels at green pixel sampling points in a row where red pixels are located in some embodiments of the present invention;
[0055] Fig. 20 Schematic diagram of a vertical interpolation filter with missing red pixels at green pixel sampling points in a row where red pixels are located in some embodiments of the present invention. DETAILED DESCRIPTION
[0056] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be understood by people with general skills in the field to which the present invention belongs. "Including" and similar words used in this article mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects.
[0057] In view of the problems existing in the prior art, an embodiment of the present invention provides a de-mosaicing method. Figure 1 , the demosaicing method comprises the following steps:
[0058] S1: Obtain the pixel gradient of each G channel pixel in the neighborhood of the target pixel in the grayscale image;
[0059] S2: Determine the interpolation point gradient according to the pixel gradient;
[0060] S3: converting the interpolation point gradient into a weight;
[0061] S4: Obtain the pixel estimation value of the target pixel;
[0062] S5: obtaining an interpolation result according to the weight and the pixel estimation value. The execution order of step S4, step S1, step S2 and step S3 is not limited, and only one embodiment is given here.
[0063] In some embodiments, the grayscale image is a Bayer domain image of size m×n, and before executing the step of obtaining the pixel gradient of each G channel pixel in the neighborhood of the target pixel in the grayscale image, the method further includes: determining the pixel array of the grayscale image; determining the color channel of the target pixel according to the pixel array, wherein the color channel includes an R channel, a G channel, and a B channel. The arrangement of the pixel array includes a CFA array, and the arrangement of the CFA array of the grayscale image includes but is not limited to an RGGB array, an RGBG array, and an RBGG array. The Chinese name of the CFA array is a color filter array, and its full English name is Color Filter Array.
[0064] In some embodiments, taking the RGGB array as an example, odd rows and odd columns are R channels, even rows and even columns are B channels, odd rows and even columns are G channels, and even rows and odd columns are G channels.
[0065] In some embodiments, before executing the step of obtaining the pixel gradient of each G channel pixel point in the neighborhood of the target pixel point, it also includes: performing boundary expansion on the grayscale image. Specifically, taking the boundary of the grayscale image as the axis of symmetry, mirroring several rows or columns of pixels at the boundary of the grayscale image. The boundary expansion includes two stages. The first stage is to mirror several rows of pixels at the upper boundary and several rows of pixels at the lower boundary of the grayscale image using the upper boundary and the lower boundary of the grayscale image as the axis of symmetry respectively; the second stage is to mirror several columns of pixels at the left boundary and several rows of pixels at the right boundary of the grayscale image using the left boundary and the right boundary of the grayscale image as the axis of symmetry respectively. Among them, the order of the first stage and the second stage is not limited.
[0066] Figure 2 Schematic diagram of a CFA array for grayscale images in some embodiments of the present invention. Figure 3 Schematic diagram of a CFA array after a grayscale image is subjected to a first stage of boundary expansion in some embodiments of the present invention. Figure 4 Schematic diagram of a CFA array after the second stage boundary extension of a grayscale image in some embodiments of the present invention.
[0067] Reference Figure 2 and Figure 3 Taking the RGGB array as an example, the upper boundary a-a' of the grayscale image is used as the symmetry axis to mirror the first and second rows of the grayscale image, and the lower boundary bb' of the grayscale image is used as the symmetry axis to mirror the last and second rows of the grayscale image.
[0068] Reference Figure 2 , Figure 3 and Figure 4 , taking the left boundary c-c' of the grayscale image as the symmetry axis, mirroring the first and second columns of the grayscale image, and taking the right boundary d-d' of the grayscale image as the symmetry axis, mirroring the last and second columns of the grayscale image.
[0069] In some embodiments, the target pixel neighborhood size is 9×9, i.e., 81 pixels, and the pixel gradient includes a horizontal pixel gradient and a vertical pixel gradient, and the horizontal pixel gradient is calculated by a horizontal pixel gradient formula, and the vertical pixel gradient is calculated by a vertical pixel gradient formula. The horizontal pixel gradient formula is: ΔH(i, j)=[2M(i, j)-M(i, j-2)-M(i, j+2)], ΔH(i, j) represents the horizontal pixel gradient of the pixel in the i-th row and j-th column, M(i, j) represents the pixel value of the pixel in the i-th row and j-th column, M(i, j-2) represents the pixel value of the pixel in the i-th row and j-2nd column, and M(i, j+2) represents the pixel value of the pixel in the i-th row and j+2nd column. The vertical pixel gradient formula is ΔV(i, j) = [2M(i, j)-M(i-2, j)-M(i+2, j)], where ΔV(i, j) represents the pixel value of the pixel in the i-th row and j-th column, M(i, j) represents the pixel value of the pixel in the i-th row and j-th column, M(i-2, j) represents the pixel value of the pixel in the i-2th row and j-th column, and M(i+2, j) represents the pixel value of the pixel in the i+2th row and j-th column.
[0070] In some embodiments, the interpolation point gradient includes a horizontal interpolation point gradient and a vertical interpolation point gradient, and determining the interpolation point gradient according to the pixel gradient includes: performing a convolution operation on all the horizontal pixel gradients and the interpolation parameter to obtain the horizontal interpolation point gradient; performing a convolution operation on all the vertical pixel gradients and the interpolation parameter to obtain the vertical interpolation point gradient. Taking the neighborhood size of 9×9 as an example, if the current interpolation point is the R channel or the B channel, the interpolation parameter is If the current interpolation point is the G channel, the interpolation parameters are
[0071] In some embodiments, converting the interpolation point gradient into a weight includes: nonlinearly mapping the difference between the horizontal interpolation point gradient and the vertical interpolation point gradient to the interval [0, 1] through a logic function to obtain a weight. The logic function is Among them, w h represents the weight, v h represents the horizontal interpolation point gradient, v v Represents the vertical interpolation point gradient.
[0072] In some embodiments, obtaining the pixel estimation value of the target pixel point includes: utilizing the principle of chromatic aberration to obtain the pixel estimation value of the target pixel point, wherein the pixel estimation value includes a horizontal pixel estimation value and a vertical pixel estimation value, and correcting the horizontal pixel estimation value and the vertical pixel estimation value by a Laplace gradient correction term.
[0073] Figure 5 FIG. 1 is a schematic diagram of a horizontal interpolation filter for missing green pixels at blue pixel sampling points in some embodiments of the present invention. Figure 5 , taking the RGGB array as an example, taking the missing green pixel at the blue pixel sampling point as an example, the calculation formula of the horizontal direction estimation value is: The horizontal pixel estimation value is corrected by the Laplace gradient correction term, then
[0074] represents the horizontal pixel estimation value of the pixel in the i-th row and j-th column, B(i, j) represents the pixel value of the pixel in the i-th row and j-th column, B(i, j-2) represents the pixel value of the pixel in the i-th row and j-2 column, B(i, j+2) represents the pixel value of the pixel in the i-th row and j+2 column, G(i, j-1) represents the pixel value of the pixel in the i-th row and j-1 column, G(i, j+1) represents the pixel value of the pixel in the i-th row and j+1 column, G(i, j-3) represents the pixel value of the pixel in the i-th row and j-3 column, and G(i, j+3) represents the pixel value of the pixel in the i-th row and j+3 column.
[0075] Figure 6 Schematic diagram of vertical interpolation filtering of missing green pixels at blue pixel sampling points in some embodiments of the present invention. Figure 6 , taking the RGGB array as an example, taking the missing green pixel at the blue pixel sampling point as an example, the vertical pixel estimation value is corrected by the Laplace gradient correction term, and the calculation formula of the vertical estimation value is:
[0076] In some embodiments, taking the RGGB array as an example, taking the missing green pixel at the blue pixel sampling point as an example, then w v =(1-w h ), g b (i,j) represents the interpolation result.
[0077] Figure 7 FIG. 1 is a schematic diagram of a horizontal interpolation filter for missing red pixels at blue pixel sampling points in some embodiments of the present invention. Figure 7 , taking the RGGB array as an example, taking the missing red pixel at the blue pixel sampling point as an example, the horizontal pixel estimation value is corrected by the Laplace gradient correction term, and the calculation formula of the horizontal estimation value is:
[0078] Figure 8 Schematic diagram of a vertical interpolation filter for missing red pixels at blue pixel sampling points in some embodiments of the present invention. Figure 8 , taking the RGGB array as an example, taking the missing red pixel at the blue pixel sampling point as an example, the vertical pixel estimation value is corrected by the Laplace gradient correction term, and the calculation formula of the vertical estimation value is:
[0079] Fig. 9 FIG. 1 is a schematic diagram of a horizontal interpolation filter for missing green pixels at red pixel sampling points in some embodiments of the present invention. Fig. 9 , taking the RGGB array as an example, taking the missing green pixel at the red pixel sampling point as an example, the horizontal pixel estimation value is corrected by the Laplace gradient correction term, and the calculation formula of the horizontal estimation value is:
[0080] Fig.10 Schematic diagram of a vertical interpolation filter for missing green pixels at red pixel sampling points in some embodiments of the present invention. Fig.10 , taking the RGGB array as an example, taking the missing green pixel at the red pixel sampling point as an example, the vertical pixel estimation value is corrected by the Laplace gradient correction term, and the calculation formula of the vertical estimation value is:
[0081] Fig.11 FIG. 1 is a schematic diagram of a horizontal interpolation filter for missing blue pixels at red pixel sampling points in some embodiments of the present invention. Fig.11, taking the RGGB array as an example, taking the missing blue pixel at the red pixel sampling point as an example, the horizontal pixel estimation value is corrected by the Laplace gradient correction term, and the calculation formula of the horizontal estimation value is:
[0082] Fig.12 Schematic diagram of a vertical interpolation filter for missing blue pixels at red pixel sampling points in some embodiments of the present invention. Fig.12 , taking the RGGB array as an example, taking the missing blue pixel at the red pixel sampling point as an example, the vertical pixel estimation value is corrected by the Laplace gradient correction term, and the calculation formula of the vertical estimation value is:
[0083] Fig.13 FIG. 1 is a schematic diagram of a horizontal interpolation filter in which a blue pixel is missing at a green pixel sampling point in a row where a blue pixel is located in some embodiments of the present invention. Fig.13 , taking the RGGB array as an example, taking the missing blue pixel at the green pixel sampling point in the row where the blue pixel is located as an example, the horizontal pixel estimation value is corrected by the Laplace gradient correction term, and the calculation formula of the horizontal estimation value is:
[0084] Fig.14 FIG. 1 is a schematic diagram of a vertical interpolation filter for missing blue pixels at green pixel sampling points in the row where the blue pixels are located in some embodiments of the present invention. Fig.14 , taking the RGGB array as an example, taking the missing blue pixel at the green pixel sampling point in the row where the blue pixel is located as an example, the vertical pixel estimation value is corrected by the Laplace gradient correction term, and the calculation formula of the vertical estimation value is:
[0085] Fig.15 FIG. 1 is a schematic diagram of a horizontal interpolation filter in which a red pixel is missing at a green pixel sampling point in a row where a blue pixel is located in some embodiments of the present invention. Fig.15 , taking the RGGB array as an example, taking the missing red pixel at the green pixel sampling point in the row where the blue pixel is located as an example, the horizontal pixel estimation value is corrected by the Laplace gradient correction term, and the calculation formula of the horizontal estimation value is:
[0086] Fig.16FIG. 1 is a schematic diagram of a vertical interpolation filter with missing red pixels at green pixel sampling points in the row where blue pixels are located in some embodiments of the present invention. Fig.16 , taking the RGGB array as an example, taking the missing red pixel at the green pixel sampling point in the row where the blue pixel is located as an example, the vertical pixel estimation value is corrected by the Laplace gradient correction term, and the calculation formula of the vertical estimation value is:
[0087] Fig.17 FIG. 1 is a schematic diagram of a horizontal interpolation filter in which a blue pixel is missing at a green pixel sampling point in a row where a red pixel is located in some embodiments of the present invention. Fig.17 , taking the RGGB array as an example, taking the missing blue pixel at the green pixel sampling point in the row where the red pixel is located as an example, the horizontal pixel estimation value is corrected by the Laplace gradient correction term, and the calculation formula of the horizontal estimation value is:
[0088] Fig.18 FIG. 1 is a schematic diagram of a vertical interpolation filter in which a blue pixel is missing at a green pixel sampling point in a row where a red pixel is located in some embodiments of the present invention. Fig.18 , taking the RGGB array as an example, taking the missing blue pixel at the green pixel sampling point in the row where the red pixel is located as an example, the vertical pixel estimation value is corrected by the Laplace gradient correction term, and the calculation formula of the vertical estimation value is:
[0089] Fig.19 FIG. 1 is a schematic diagram of a horizontal interpolation filter with missing red pixels at green pixel sampling points in the row where red pixels are located in some embodiments of the present invention. Fig.19 , taking the RGGB array as an example, taking the missing red pixel at the green pixel sampling point in the row where the red pixel is located as an example, the horizontal pixel estimation value is corrected by the Laplace gradient correction term, and the calculation formula of the horizontal estimation value is:
[0090] Fig. 20 FIG. 1 is a schematic diagram of a vertical interpolation filter for missing red pixels at green pixel sampling points in the row where red pixels are located in some embodiments of the present invention. Fig. 20 , taking the RGGB array as an example, taking the missing red pixel at the green pixel sampling point in the row where the red pixel is located as an example, the vertical pixel estimation value is corrected by the Laplace gradient correction term, and the calculation formula of the vertical estimation value is:
[0091] Peak Signal to Noise Ratio (PSNR) is often used as a detection method for signal reconstruction quality in fields such as image compression. It is often simply defined by the mean square error. For two m×n monochrome images J(i, j) and K(i, j), if one is a noise approximation of the other, then their mean square error formula is: The peak signal-to-noise ratio is calculated as: Where MSE stands for mean square error, and PSNR stands for peak signal-to-noise ratio. The grayscale image is interpolated and reconstructed by the bilinear interpolation algorithm, and the PSNR of the reconstructed image is about 3.6a, where a is greater than 0; the grayscale image is interpolated and reconstructed by the gradient interpolation method, and the PSNR of the reconstructed image is about 4.4a; the grayscale image is interpolated and reconstructed by this application, and the PSNR of the reconstructed image is about 4.7a. It can be seen that the PSNR of the image reconstructed by this application is higher.
[0092] The gradient interpolation algorithm needs to interpolate the green channel pixels first during the interpolation process, and the interpolation of the red and blue channel pixels needs to refer to the green channel pixels after the interpolation is completed. When the hardware circuit is implemented, more additional resources are consumed, and the operation speed will be significantly reduced. However, the present application utilizes the principle of chromatic aberration for color restoration, and also interpolates by referring to the gradient changes of surrounding pixels, which can better restore the color and texture details of the image, making the interpolation result closer to reality. In addition, when the red, green and blue three channels are interpolated, the present application can be performed simultaneously with a high degree of parallelism, which greatly saves hardware resources during circuit implementation, improves the calculation speed, and is also more suitable for real-time color restoration of ultra-high-definition images.
[0093] The present invention also provides a system for implementing the de-mosaicing method, the system comprising a pixel gradient acquisition unit, an interpolation point gradient determination unit, a weight conversion unit, a pixel estimation value acquisition unit and an interpolation result calculation unit, the pixel gradient acquisition unit being used to acquire the pixel gradient of each G channel pixel point in the neighborhood of the target pixel point in the grayscale image; the interpolation point gradient determination unit being used to determine the interpolation point gradient according to the pixel gradient; the weight conversion unit being used to convert the interpolation point gradient into a weight; the pixel estimation value acquisition unit being used to acquire the pixel estimation value of the target pixel point; and the interpolation result calculation unit being used to obtain the interpolation result according to the weight and the pixel estimation value.
[0094] The system also includes an arrangement determination unit, a color channel determination unit and a boundary extension unit. The arrangement determination unit is used to determine the arrangement of the CFA array of the grayscale image. The color channel determination unit is used to determine the color channel of the target pixel point according to the arrangement of the CFA array. The color channel includes an R channel, a G channel and a B channel. The boundary extension unit is used to extend the boundary of the grayscale image.
[0095] The present invention also provides a device, comprising:
[0096] processor;
[0097] a memory for storing processor-executable instructions;
[0098] Wherein, when the processor is configured to execute instructions in the memory, the de-mosaicing method is implemented.
[0099] The present invention also provides a computer-readable storage medium, comprising computer instructions, and when the computer instructions are executed by a processor, the de-mosaicing method is implemented.
[0100] Computer readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or components, or any combination thereof. More specific examples of computer readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM) or flash memory, an optical fiber, a portable compact disk read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device.
[0101] The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wire, optical cable, or any suitable combination of the foregoing.
[0102] Computer program code for performing the operations of the present specification may be written in one or more programming languages or a combination thereof, such as Java, Smalltalk, C++, C, etc. The program code may execute entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server.
[0103] Although the embodiments of the present invention are described in detail above, it is obvious to those skilled in the art that various modifications and variations can be made to these embodiments. However, it should be understood that such modifications and variations are within the scope and spirit of the present invention as described in the claims. Moreover, the present invention described herein may have other embodiments and may be implemented or realized in a variety of ways.
Claims
1. A demosaicing method, characterized in that: include: Get the pixel gradient of each G channel pixel in the neighborhood of the target pixel in the grayscale image; Determine an interpolation point gradient according to the pixel gradient; Converting the interpolation point gradient into a weight; Get the pixel estimation value of the target pixel; An interpolation result is obtained according to the weight and the pixel estimation value.
2. The demosaicing method according to claim 1, characterized in that: Before executing the step of obtaining the pixel gradient of each G channel pixel in the neighborhood of the target pixel, the method further includes: determining a pixel array of a grayscale image; The color channels of the target pixel are determined according to the pixel array, and the color channels include an R channel, a G channel, and a B channel.
3. The demosaicing method according to claim 1 or 2, characterized in that: Before executing the step of obtaining the pixel gradient of each G channel pixel in the neighborhood of the target pixel, the method further includes: Perform boundary extension on a grayscale image.
4. The demosaicing method according to claim 3, characterized in that: The step of extending the boundary of the grayscale image comprises: Taking the boundary of the grayscale image as the axis of symmetry, a plurality of rows or columns of pixel points at the boundary of the grayscale image are mirrored.
5. The demosaicing method according to claim 1, characterized in that: The pixel gradient includes a horizontal pixel gradient and a vertical pixel gradient, the interpolation point gradient includes a horizontal interpolation point gradient and a vertical interpolation point gradient, and determining the interpolation point gradient according to the pixel gradient includes: Performing a convolution operation on all the horizontal pixel gradients and the interpolation parameter to obtain the horizontal interpolation point gradient; A convolution operation is performed on all the vertical pixel gradients and the interpolation parameter to obtain the vertical interpolation point gradient.
6. The demosaicing method according to claim 1, characterized in that: The interpolation point gradient includes a horizontal interpolation point gradient and a vertical interpolation point gradient, and converting the interpolation point gradient into a weight includes: The difference between the horizontal interpolation point gradient and the vertical interpolation point gradient is nonlinearly mapped to the interval [0, 1] through a logic function to obtain a weight.
7. The demosaicing method according to claim 6, characterized in that: The logic function is Among them, w h represents the weight, v h represents the horizontal interpolation point gradient, v v Represents the vertical interpolation point gradient.
8. The demosaicing method according to claim 1, characterized in that: The step of obtaining a pixel estimation value of a target pixel point includes: The pixel estimation value of the target pixel point is obtained by using the color difference principle, and the pixel estimation value includes a horizontal pixel estimation value and a vertical pixel estimation value.
9. The demosaicing method according to claim 8, characterized in that: The step of obtaining the pixel estimation value of the target pixel point further includes: The horizontal pixel estimation value and the vertical pixel estimation value are corrected by a Laplace gradient correction term.
10. A system for implementing the de-mosaicing method according to any one of claims 1 to 9, characterized in that: The system comprises a pixel gradient acquisition unit, an interpolation point gradient determination unit, a weight conversion unit, a pixel estimation value acquisition unit and an interpolation result calculation unit, wherein the pixel gradient acquisition unit is used to acquire the pixel gradient of each G channel pixel point in the neighborhood of the target pixel point in the grayscale image; the interpolation point gradient determination unit is used to determine the interpolation point gradient according to the pixel gradient; the weight conversion unit is used to convert the interpolation point gradient into a weight; the pixel estimation value acquisition unit is used to acquire the pixel estimation value of the target pixel point; and the interpolation result calculation unit is used to obtain the interpolation result according to the weight and the pixel estimation value.
11. The system according to claim 10, characterized in that It also includes an arrangement determination unit, a color channel determination unit and a boundary extension unit. The arrangement determination unit is used to determine the arrangement of the CFA array of the grayscale image. The color channel determination unit is used to determine the color channel of the target pixel point according to the arrangement of the CFA array. The color channel includes an R channel, a G channel and a B channel. The boundary extension unit is used to extend the boundary of the grayscale image.
12. A device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, when the processor is configured to execute the instructions of the memory, the de-mosaic method according to any one of claims 1 to 9 is implemented.
13. A computer-readable storage medium, characterized in that: The method comprises computer instructions, and when the computer instructions are executed by a processor, the de-mosaicing method according to any one of claims 1 to 9 is implemented.
Citation Information
Cited By
Data processing method, data processing device, electronic equipment and storage medium
CN122222811A
