Image processing method, device and computer readable storage medium

Abstract

Embodiments of the present application provide an image processing method, device and computer readable storage medium, the method comprising: performing W interpolation on HexRGBW data obtained by a HexRGBW configuration color filter array to generate a full-resolution W image; performing first sampling processing on the full-resolution W image and the HexRGBW data to generate a half-resolution W image and a half-resolution RGB image; performing RGB interpolation processing on the half-resolution RGB image according to the half-resolution W image to generate a half-resolution R image, a half-resolution G image and a half-resolution B image; and performing second sampling processing on the half-resolution R image, the half-resolution G image and the half-resolution B image to generate a full-resolution RGB image. Embodiments of the present application can perform demosaicing processing on a HexRGBW configuration color filter array.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to image processing methods, apparatus and computer-readable storage media. Background Technology

[0002] Image sensors are used to capture images, and for cost reasons, they typically employ a single CCD or CMOS image sensor covered with a color filter array (CFA). Each pixel of this image sensor has only information from a single channel, and the three-channel information of each pixel is restored through a specific demosaic algorithm.

[0003] Color filter arrays primarily come in Bayer, Quad Bayer, and RGBW configurations. The HexRGBW configuration is a type of RGBW color filter array. HexRGBW color filter arrays possess more luminance / grayscale information, allowing them to maintain correct color saturation while accurately perceiving grayscale information even in low-light environments. Therefore, how to de-mosaicize HexRGBW color filter arrays has become a pressing technical problem to be solved. Summary of the Invention

[0004] This application provides an image processing method, apparatus, and computer-readable storage medium capable of de-mosaicing a HexRGBW color filter array.

[0005] The first aspect of this application provides an image processing method, the method comprising: performing W interpolation on HexRGBW data obtained from a HexRGBW configuration color filter array to generate a full-resolution W image; performing a first sampling process on the full-resolution W image and the HexRGBW data to generate a half-resolution W image and a half-resolution RGB image; performing RGB interpolation on the half-resolution RGB image based on the half-resolution W image to generate a half-resolution R image, G image, and B image; and performing a second sampling process on the half-resolution R image, G image, and B image to generate a full-resolution RGB image.

[0006] A second aspect of the present invention provides an image processing apparatus, comprising: a W image generation module, configured to perform W interpolation on HexRGBW data obtained from a HexRGBW configuration color filter array to generate a full-resolution W image; a first sampling module, configured to perform a first sampling process based on the full-resolution W image and the HexRGBW data to generate a half-resolution W image and a half-resolution RGB image; an RGB interpolation module, configured to perform RGB interpolation process based on the half-resolution W image and the half-resolution RGB image to generate a half-resolution R image, G image, and B image; and a second sampling module, configured to perform a second sampling process based on the half-resolution R image, G image, and B image to generate a full-resolution RGB image.

[0007] In a third aspect, this invention provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the image processing method described above.

[0008] This application addresses the issue of sparse R, G, and B pixel distribution in HexRGBW color filter arrays by employing uniformly and densely distributed W channels for W interpolation to obtain a full-resolution W image. Then, a first sampling process is performed on the full-resolution W image and the HexRGBW data to generate a half-resolution W image and a half-resolution RGB image with densely distributed R, G, and B pixels. Based on the half-resolution W image, RGB interpolation is performed on the half-resolution RGB image to generate half-resolution R, G, and B images. Finally, a second sampling process generates the full-resolution RGB image. Therefore, this application effectively reduces the computational and spatial consumption required for de-mosaic processing of HexRGBW color filter arrays. Attached Figure Description

[0009] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments conforming to the invention and, together with the description, serve to explain the technical solutions of the invention. Some specific embodiments of the invention will be described in detail below by way of example and not limitation, with reference to the accompanying drawings. The same reference numerals in the drawings denote the same or similar parts or components. Those skilled in the art should understand that these drawings are not necessarily drawn to scale.

[0010] Figure 1 A schematic diagram of an image processing apparatus provided for an embodiment of this application;

[0011] Figure 2 This is a schematic diagram of the pixel array of the color filter array of the HexRGBW configuration in an embodiment of this application;

[0012] Figure 3 A flowchart illustrating the steps of an image processing method provided in this application embodiment;

[0013] Figure 4 A flowchart of step S1 in an image processing method provided in this application embodiment;

[0014] Figure 5 This is a schematic diagram illustrating W interpolation of HexRGBW data according to an embodiment of this application;

[0015] Figure 6 A flowchart of step S2 in an image processing method provided in an embodiment of this application;

[0016] Figure 7 This is a schematic diagram illustrating the arithmetic mean of the W values ​​in the diagonal directions to obtain a downsampled half-resolution W image according to an embodiment of this application.

[0017] Figure 8 This is a schematic diagram illustrating the arithmetic averaging of the R, G, and B values ​​in the diagonal directions to obtain a downsampled Quad Bayer four-pixel image according to an embodiment of this application.

[0018] Figure 9 A flowchart of step S3 in an image processing method provided in this application embodiment;

[0019] Figure 10 This is a schematic diagram of the horizontal, vertical, diagonal, and oblique diagonal directions of the embodiments of this application;

[0020] Figure 11 This is a schematic diagram illustrating step S3 of the embodiments of this application, which generates half-resolution R, G, and B images.

[0021] Figure 12 A flowchart of step S31 in an image processing method provided in this application embodiment;

[0022] Figures 13-16 Schematic diagrams are shown for calculating the gradient in the horizontal, vertical, diagonal, and diagonal directions, respectively.

[0023] Figure 17 A flowchart of step S32 in an image processing method provided in this application embodiment;

[0024] Figure 18 This is a schematic diagram of Quad Bayer pixels provided in an embodiment of this application;

[0025] Figures 19(A) to 19(P) This is a schematic diagram of the pixels involved in interpolation in the horizontal, vertical, diagonal, and oblique directions at the top left, top right, bottom left, and bottom right corners of an embodiment of this application.

[0026] Figure 20 A flowchart of step S33 in an image processing method provided in this application embodiment;

[0027] Figure 21 A flowchart illustrating step S34 of an image processing method provided in this application embodiment;

[0028] Figure 22 (A) to 22(P) are schematic diagrams of pixels involving interpolation in the horizontal, vertical, diagonal and oblique directions of the upper left corner, upper right corner, lower left corner and lower right corner of the embodiment of this application;

[0029] Figure 23 A flowchart of step S35 in an image processing method provided in an embodiment of this application;

[0030] Figure 24 A flowchart illustrating step S36 in an image processing method provided in this application embodiment;

[0031] Figure 25 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. Detailed Implementation

[0032] The technical solutions in the embodiments of the present invention will now be described with reference to the accompanying drawings. Various details are set forth in this invention because these details relate to certain embodiments. However, the invention may also be implemented in a manner different from that described herein. Modifications to the discussed embodiments can be made by those skilled in the art without departing from the invention. Therefore, the invention is not limited to the specific embodiments disclosed herein.

[0033] Figure 1 A schematic diagram of an image processing apparatus 10 is shown. This image processing apparatus 10 can be any electronic device, or it can be part of an electronic device, such as a camera module within an electronic device; however, the embodiments of this application are not limited thereto. Figure 1 As shown, the image processing apparatus 10 includes a pixel array 100 and a processor 200. The image processing apparatus 10 utilizes the photoelectric conversion function of the pixel array 100 to convert incident light related to the imaging object into an electrical signal. The electrical signal is proportional to the intensity or brightness of the incident light, and the processor 200 processes the electrical signal to obtain an image of the imaging object.

[0034] Figure 2This is a schematic diagram of a pixel array 100 corresponding to a HexRGBW color filter array. The pixel array 100 includes multiple pixels 111 arranged in an array. The multiple pixels 111 include: red pixel 111-a, green pixel 111-b, blue pixel 111-c, and white pixel 111-d. Light incident on the pixel array 100 typically includes multiple components with different wavelengths. Red pixel 111-a is configured to sense the red component of the incident light, green pixel 111-b is configured to sense the green component of the incident light, and blue pixel 111-c is configured to sense the blue component of the incident light. The red, green, and blue components correspond to different wavelength bands of light. White pixel 111-d is configured to sense light across all wavelength bands. The incident light information sensed by red pixel 111-a is called red channel information or R value; the incident light information sensed by green pixel 111-b is called green channel information or G value; the incident light information sensed by blue pixel 111-c is called blue channel information or B value; and the incident light information sensed by white pixel 111-d represents the brightness of the incident light and is called W value. Red pixel 111-a, green pixel 111-b, and blue pixel 111-c can only directly obtain one of the corresponding R, G, and B values. To achieve color display, a demosaicing algorithm is needed to calculate the G and B values ​​for red pixel 111-a, the R and B values ​​for green pixel 111-b, the R and G values ​​for blue pixel 111-b, and the R, G, and B values ​​for white pixel 111-d.

[0035] Figure 2 An 8x8 pixel array 111 is shown, which represents a periodic unit of the pixel array 100 corresponding to the HexRGBW arrangement. Within this periodic unit, white pixels occupy 1 / 2, green pixels occupy 1 / 4, and red and blue pixels each occupy 1 / 8. The pixel array 100 includes multiple such periodic units. Further, as... Figure 2 As shown, the periodic unit includes: one sub-unit composed of red pixel 111-a and white pixel 111-d, one sub-unit composed of blue pixel 111-c and white pixel 111-d, and two sub-units composed of green sub-pixels 111-b and white pixels 111-d.

[0036] This application provides an image processing method, see [link to relevant documentation] Figure 3 The method includes:

[0037] Step S1: Perform W interpolation on the HexRGBW data obtained from the HexRGBW configuration color filter array to generate a full-resolution W image. The HexRGBW data is a full-resolution image obtained from the pixel array 100. For example, the pixel array 100 includes m rows and n columns of pixels, and the HexRGBW data is a collection of information collected for each pixel. In the pixel array 100 of the HexRGBW configuration color filter array, 50% are white pixels and 50% are non-white pixels. For each non-white pixel, the W value of the non-white pixel is calculated using interpolation based on the W values ​​of its surrounding white pixels (or interpolated related white pixels). This yields the W value for each pixel in the pixel array 100, thus generating a full-resolution W image.

[0038] Step S2: Perform a first sampling process based on the full-resolution W image and the HexRGBW data to generate a half-resolution W image and a half-resolution RGB image, respectively. The pixel array 100 includes m rows and n columns of pixels. The full-resolution W image includes m rows and n columns of pixels. For example, the pixels in the full-resolution W image correspond one-to-one with the pixels in the pixel array 100. Both the half-resolution W image and the half-resolution RGB image include (m / 2) rows and (n / 2) columns of pixels. For example, the pixels in the half-resolution W image and the half-resolution RGB image correspond to 2x2 pixels in the pixel array 100. The half-resolution RGB image has a Quad Bayer configuration.

[0039] Step S3: Perform RGB interpolation on the half-resolution RGB image based on the half-resolution W image to generate half-resolution R, G, and B images. All pixels in the half-resolution R image are R pixels, all pixels in the half-resolution G image are G pixels, and all pixels in the half-resolution B image are B pixels.

[0040] Step S4: Perform a second sampling process on the half-resolution R image, G image, and B image to generate a full-resolution RGB image. Specifically, perform a second sampling process on the half-resolution R image, G image, and B image to generate a full-resolution R image, a full-resolution G image, and a full-resolution B image; merge the full-resolution R image, full-resolution G image, and full-resolution B image into a full-resolution RGB image. Each pixel in the full-resolution RGB image has an R value, a G value, and a B value.

[0041] This application addresses the issue of sparse R, G, and B pixel distribution in HexRGBW color filter arrays by performing W interpolation on the uniformly and densely distributed W channels to obtain a full-resolution W image. Then, a first sampling process is performed on the full-resolution W image and the HexRGBW data to generate a half-resolution W image and a half-resolution RGB image containing only R, G, and B pixels. Based on the half-resolution W image, RGB interpolation is performed on the half-resolution RGB image to generate half-resolution R, G, and B images. Finally, a second sampling process is performed to generate the full-resolution RGB image. Therefore, this application effectively reduces the computational and spatial consumption required for de-mosaic processing of HexRGBW color filter arrays.

[0042] In the specific implementation of Embodiment 1 of this application, see [link to implementation details]. Figure 4 Step S1 includes:

[0043] Step S11: Obtain the horizontal gradient gradH and vertical gradient gradV of the W value in the HexRGBW data for non-white pixels.

[0044] Step S12: If the horizontal gradient gradH is less than the first threshold and the vertical gradient gradV is less than the second threshold, determine the W value of the non-white pixel by isotropic interpolation.

[0045] Step S13: If the absolute value of the difference between the horizontal gradient gradH and the vertical gradient gradV is less than the third threshold, the W value of the non-white pixel is determined by isotropic interpolation.

[0046] Step S14: If the absolute value of the difference between the horizontal gradient gradH and the vertical gradient gradV is greater than the third threshold, and the horizontal gradient gradH is less than the vertical gradient gradV, the W value of the non-white pixel is determined by horizontal interpolation; if the absolute value of the difference between the horizontal gradient gradH and the vertical gradient gradV is greater than the third threshold, and the horizontal gradient gradH is greater than the vertical gradient gradV, the W value of the non-white pixel is determined by vertical interpolation.

[0047] Specifically, see Figure 5 The position of a pixel in pixel array 100 is represented by the horizontal coordinate axis j and the vertical coordinate axis i, and the coordinate offset is marked with numbers. C represents an R pixel, G pixel, or B pixel, i.e., a non-white pixel. The C pixel marked with a circle is the target pixel, i.e., the pixel to be interpolated. i，j W represents the target pixel to be interpolated. i，jThis represents the W value of non-white pixels predicted through interpolation. In this embodiment, the W values ​​of white pixels with coordinate offsets between -2 and +2 are used for horizontal gradient gradH, vertical gradient gradV, and isotropic interpolation. Horizontal interpolation and vertical interpolation The calculation of W. i，j Based on steps S11 to S14, isotropic interpolation is performed. Horizontal interpolation and vertical interpolation One of them.

[0048] The formulas for calculating the horizontal gradient gradH and the vertical gradient gradV are given in formulas (1) and (2), respectively.

[0049] gradH = |W i，j-1 -W i，j+1 |+|W i-1，j-2 -W i-1，j |+|W i-1，j+2 -W i-1，j |+|W i+1，j-2 -W i+1，j |+|W i+1，j+2 -W i+1，j | (1)

[0050] gradV=|W i-1，j -W i+1，j |+|Wi -2，j-1 -W i，j-1 |+|W i+2，j-1 -W i，j-1 |+|W i-2，j+1 -W i，j+1 |+|W i+2，j+1 -W i，j+1 | (2)

[0051] For example, pixel C 0，0 The horizontal gradient gradH at point C is calculated based on the W values ​​of eight white pixels with coordinates (0, -1), (0, 1), (-1, -2), (-1, 0), (-1, 2), (1, -2), (1, 0), and (1, 2). Specifically, these eight white pixels are located at pixel C. 0,0 The row and pixel C 0,0 The row preceding the current row, pixel C 0,0 The next line after the current line. Based on these 8 pixels, 5 pairs of white pixels were selected. The two pixels in each pair of white pixels are in the same row, separated by a non-white pixel.

[0052] Pixel C0,0 The vertical gradient gradV at point C is calculated based on the W values ​​of eight white pixels with coordinates (-1, 0), (1, 0), (-2, -1), (0, -1), (2, -1), (-2, 1), (0, 1), and (2, 1). Specifically, these eight white pixels are located at pixel C. 0,0 The column and pixel C 0,0 The column preceding the current column, pixel C 0,0 The next column after the current column. Based on these 8 pixels, 5 pairs of white pixels were selected. The two pixels in each pair of white pixels are in the same column and are separated by a non-white pixel.

[0053] Isotropic interpolation Horizontal interpolation and vertical interpolation The calculation formulas are shown in Formula (3), Formula (4) and Formula (5), respectively. The first term of Formula (3), Formula (4) and Formula (5) is the low-frequency component, and the other terms are the high-frequency components.

[0054]

[0055]

[0056]

[0057] Specifically, the first threshold, the second threshold, and the third threshold are constant thresholds, which can be set by those skilled in the art as needed.

[0058] The embodiments of this application provide isotropic interpolation for the W value. Horizontal interpolation and vertical interpolation Three interpolation methods are used to restore the W value by adding low-frequency components to high-frequency components, which can effectively restore image details and reduce noise.

[0059] In another specific implementation of the embodiments of this application, see [link to relevant documentation]. Figure 6 Step S2 includes:

[0060] Step S21: The arithmetic mean of the W values ​​in the diagonal direction of the full-resolution W image is used to obtain the downsampled half-resolution W image.

[0061] Specifically, see Figure 7In this embodiment, the W values ​​in the diagonal directions are arithmetically averaged to obtain a downsampled half-resolution W image. The full-resolution W image includes multiple 2x2 pixel subarrays, each corresponding to a pixel in the half-resolution W image. The arithmetic mean of the W values ​​of the top-left and bottom-right pixels in each 2x2 pixel subarray is used as the W value of the corresponding pixel in the half-resolution W image. It can be understood that the method of calculating the W value of the corresponding pixel in the half-resolution W image based on the W values ​​of the 2x2 pixels in the full-resolution W image is not limited to this.

[0062] Step S22: Perform an arithmetic average of the R, G, and B values ​​in the diagonal direction of the HexRGBW data to obtain a downsampled half-resolution RGB image.

[0063] Specifically, see Figure 8 In this embodiment, the half-resolution RGB image obtained by arithmetically averaging the R, G, and B values ​​in the diagonal directions is a Quad Bayer image. The HexRGBW data also includes multiple 2x2 pixel subarrays. In these 2x2 subarrays, the pixels in the lower left and upper right corners are white pixels, and the pixels in the upper left and lower right corners are either all red, all green, or all blue pixels. The arithmetic mean of the pixels in the upper left and lower right corners is used as the pixel value for the corresponding pixel in the half-resolution RGB image.

[0064] In another specific implementation of the embodiments of this application, see [link to relevant documentation]. Figure 9 Step S3 includes:

[0065] Step S31: Perform gradient calculation based on the half-resolution image W to obtain the gradient values ​​of the half-resolution image W in multiple directions, including horizontal, vertical, diagonal, and oblique diagonal directions. Specifically, calculate the gradient values ​​in multiple directions for each pixel in the half-resolution image W. Calculate the gradient values ​​in the horizontal, vertical, diagonal, and oblique diagonal directions based on the four adjacent pixels of the target pixel in each of the horizontal, vertical, diagonal, and oblique diagonal directions.

[0066] Specifically, see Figure 10 The horizontal, vertical, diagonal, and oblique directions are shown in the figure.

[0067] Step S32: Based on the gradient values ​​in the multiple directions and the half-resolution W image, interpolate the third pixel values ​​of the first and second pixels among the R, G, and B pixels of the half-resolution RGB image to generate a 100% third pixel half-resolution image. All pixels in the 100% third pixel half-resolution image are third pixels, and the resolution is half the resolution of the HexRGBW data.

[0068] Step S33: Based on the gradient values ​​in the multiple directions and the half-resolution W image, interpolate the first pixel value of the second pixel among the R, G, and B pixels of the half-resolution RGB image to generate a 50% first pixel half-resolution image. In the 50% first pixel half-resolution image, half of the pixels are the first pixels, and the other half are the third pixels.

[0069] Step S34: Based on the gradient values ​​in the multiple directions and the half-resolution W image, interpolate the second pixel value of the first pixel among the R, G, and B pixels of the half-resolution RGB image to generate a 50% second pixel half-resolution image. In the 50% second pixel half-resolution image, half of the pixels are second pixels, and the other half are third pixels.

[0070] Step S35: Based on the gradient values ​​in the multiple directions and the half-resolution W image, interpolate the first pixel value of the third pixel in the 50% first pixel half-resolution image to generate a 100% first pixel half-resolution image.

[0071] Step S36: Based on the gradient values ​​in the multiple directions and the half-resolution W image, interpolate the second pixel value of the third pixel in the 50% second pixel half-resolution image to generate a 100% second pixel half-resolution image.

[0072] Specifically, the third pixel is the G pixel, the first pixel is one of the R pixel and the B pixel, and the second pixel is the other of the R pixel and the B pixel.

[0073] See Figure 11 This application embodiment generates a 100% G-pixel half-resolution image by interpolating the G-pixel values ​​of the B-pixels and R-pixels in a half-resolution RGB image based on the gradient values ​​in the multiple directions and the half-resolution W image. This application embodiment also generates a 50% B-pixel half-resolution image by interpolating the B-pixel values ​​of the R-pixels in a half-resolution RGB image based on the gradient values ​​in the multiple directions and the half-resolution W image. This application embodiment further generates a 50% R-pixel half-resolution image by interpolating the R-pixel values ​​of the B-pixels in a half-resolution RGB image based on the gradient values ​​in the multiple directions and the half-resolution W image. This application embodiment also generates a 100% B-pixel half-resolution image by interpolating the G-pixels in a 50% R-pixel half-resolution image based on the gradient values ​​in the multiple directions and the half-resolution W image.

[0074] This application embodiment targets half-resolution RGB images with Quad Bayer configuration. It uses a half-resolution W image that fills the entire image to guide the interpolation of R, G, and B pixels, which can effectively suppress artifacts such as false colors and zipper effects.

[0075] In another specific implementation of the embodiments of this application, see [link to relevant documentation]. Figure 12 Step S31 includes:

[0076] Step S311: Obtain the gradient value in the horizontal direction based on the absolute value of the difference between multiple pairs of horizontal pixels related to the pixel to be interpolated in the half-resolution W image.

[0077] Step S312: Obtain the gradient value in the vertical direction based on the absolute value of the difference between multiple pairs of vertically related pixels in the half-resolution W image.

[0078] Step S313: Obtain the gradient value in the diagonal direction based on the absolute value of the difference between multiple pairs of diagonal pixels related to the pixel to be interpolated in the half-resolution W image.

[0079] Step S314: Obtain the gradient value in the diagonal direction based on the absolute value of the difference between multiple pairs of diagonal pixels related to the pixel to be interpolated in the half-resolution W image.

[0080] Specifically, see Figures 13-16 Schematic diagrams are shown for calculating gradients in the horizontal, vertical, diagonal, and diagonal directions, respectively.

[0081] The horizontal gradient grad h Vertical gradient grad v Diagonal gradient grad ad and the gradient grad in the diagonal direction bg The pixels involved in the calculation are as follows: Figure 13 , Figure 14 , Figure 15 , Figure 16 As shown, the horizontal coordinate axis is j, the vertical coordinate axis is i, the coordinate offset is marked with numbers, and the pixel marked with a circle is the target pixel, which is represented by coordinates as W. i，j The remaining pixels are the involved pixels. In this embodiment, pixels with coordinate offsets between -2 and +2 are used as the involved pixels. The gradient in each direction includes high-frequency gradient and low-frequency gradient, and their calculation is shown in Equations (6), (7), (8), and (9), respectively.

[0082] grad h =(|Wi，j-2 -IN i，j-1 |+|In i，j-1 -IN i，j |+|In i，j -IN i，j+1 |+|In i，j+1 -IN i，j+2 |+|In i，j-2 -IN i，j |+|In i，j-1 -IN i，j+1 |+|In i，j -IN i，j+2 |) / 7 (6)

[0083] hail v =(|W i-2，j -IN i-1，j |+|In i-1，j -IN i，j |+|In i，j -IN i+1，j |+|In i+1，j -IN i+2，j |+|In i-2，j -IN i，j |+|In i-1，j -IN i+1，j |+|In i，j -IN i+2，j |) / 7 (7)

[0084] hail ad =(|W i+2，j-2 -IN i+1，j-1 |+|In i+1，j-1 -IN i，j |+|In i，j -IN i-1，j+1 |+|In i-1，j+1 -IN i-2，j+2 |+|In i+2，j-2 -IN i，j |+|In i+1，j-1 -IN i-1，j+1 |+|In i，j -IN i-2，j+2 |) / 7 (8)

[0085] hail bg =(|W i-2，j-2 -IN i-1，j-1 |+|In i-1，j-1 -IN i，j |+|In i，j -IN i+1，j+1 |+|In i+1，j+1 -IN i+2，j+2 |+|In i-2，j-2 -IN i，j |+|In i-1，j-1 -INi+1，j+1 |+|W i，j -W i+2，j+2 |) / 7 (9)

[0086] In another specific implementation of the embodiments of this application, see [link to relevant documentation]. Figure 17 Step S32 includes:

[0087] Step S321: Based on the gradient values ​​in the multiple directions, determine the direction corresponding to the minimum gradient as the interpolation direction.

[0088] Step S322: Calculate the difference between the pixel value of the third pixel point related to the first pixel point and the second pixel point in the interpolation direction and the W pixel value in the half-resolution W image as the color difference value of the third pixel point.

[0089] Step S323: Calculate the weight of the third pixel related to the first pixel and the second pixel in the interpolation direction; perform weighted interpolation on the first pixel and the second pixel according to the color difference value and weight of the third pixel to generate the 100% third pixel half-resolution image.

[0090] Specifically, this application embodiment uses the example of the first pixel being a B pixel, the second pixel being an R pixel, and the third pixel being a G pixel. However, this application is not limited to this; the first pixel, the second pixel, and the third pixel can be R, G, and B pixels respectively, as needed.

[0091] See Figure 18 In a half-resolution Quad Bayer RGB image, B and R pixels are arranged in groups of four pixels. The interpolation method for the four points within a group is slightly different. Specifically, the white pixels involved in interpolation at the top-left, top-right, bottom-left, and bottom-right corners in the horizontal, vertical, diagonal, and oblique directions are as follows: Figures 19(A) to 19(P) As shown. That is, in step S322, the situation of the third pixel related to the first pixel and the second pixel in the interpolation direction is as follows. Figures 19(A) to 19(P) As shown.

[0092] Figures 19(A) to 19(P) The horizontal coordinate axis is j, and the vertical coordinate axis is i. The coordinate offset is marked with numbers. The pixel marked with a circle is the target pixel. (m,n) is the coordinate offset of the top left, top right, bottom left and bottom right corners, as shown in formula (10).

[0093] (m,n)={(0,0),(0,1),(1,0),(1,1)} (10)

[0094] The G pixel used for interpolation is represented as E. i+p,j+q , (p,q) is the coordinate offset of pixel G used for interpolation.

[0095] See Figures 19(A) to 19(D) The coordinate offset (p, q) of the G pixel used for interpolation in the horizontal direction at the top left corner is shown in formula (11):

[0096] (p,q)1={(0,-2),(0,-1),(0,2)} (11)

[0097] The coordinate offset (p, q)2 of the G pixel used for interpolation in the vertical direction at the top left corner point is shown in formula (12):

[0098] (p,q)2={(-2,0),(-1,0),(2,0)} (12)

[0099] The coordinate offset (p, q)3 of the G pixel used for interpolation in the diagonal direction of the top left corner point is shown in formula (13):

[0100] (p,q)3={(1,-1),(-1,1)} (13)

[0101] The coordinate offset (p, q) of the G pixel used for interpolation in the diagonal direction of the top left corner point is shown in formula (14):

[0102] (p,q)4={(0,-1),(-1,0),(2,1),(1,2)} (14)

[0103] See Figures 19(E) to 19(H) The coordinate offset (p, q) of the G pixel used for interpolation in the horizontal direction at the upper right corner is shown in formula (15):

[0104] (p,q)5={(0,-1),(0,2),(0,3)} (15)

[0105] The coordinate offset (p, q) of the G pixel used for interpolation in the vertical direction at the top right corner is shown in formula (16):

[0106] (p,q)6={(-2,1),(-1,1),(2,1)} (16)

[0107] The coordinate offset (p, q) of the G pixel used for interpolation in the diagonal direction at the top right corner point is shown in formula (17):

[0108] (p,q)7={(-1,1),(0,2),(1,-1),(2,0)} (17)

[0109] The coordinate offset (p, q) of the G pixel used for interpolation in the diagonal direction of the upper right corner point is shown in formula (18):

[0110] (p,q)8={(-1,0),(1,2)} (18)

[0111] See Figures 19(I) to 19(L) The coordinate offset (p, q) of the G pixel used for interpolation in the horizontal direction at the lower left corner is shown in formula (19):

[0112] (p,q)9={(1,-2),(1,-1),(1,2)} (19)

[0113] The coordinate offset (p, q) of the G pixel used for interpolation in the vertical direction at the lower left corner point is shown in formula (20):

[0114] (p,q)10={(-1,0),(2,0),(3,0)} (20)

[0115] The coordinate offset (p, q) of the G pixel used for interpolation in the diagonal direction at the lower left corner point is shown in formula (21):

[0116] (p,q)11={(-1,1),(0,2),(1,-1),(2,0)} (21)

[0117] The coordinate offset (p, q) of the G pixel used for interpolation in the diagonal direction of the lower left corner point is shown in formula (22):

[0118] (p,q)12={(0,-1),(2,1)} (22)

[0119] See Figures 19(M) to 19(P) The coordinate offset (p, q) of the G pixel used for interpolation in the horizontal direction at the lower right corner is shown in formula (23):

[0120] (p,q)13={(1,-1),(1,2),(1,3)} (23)

[0121] The coordinate offset (p, q) of the G pixel used for interpolation in the vertical direction at the bottom right corner is shown in formula (24):

[0122] (p,q)14={(-1,1),(2,1),(3,1)} (24)

[0123] The coordinate offset (p, q) of the G pixel used for interpolation in the diagonal direction at the bottom right corner is shown in formula (25):

[0124] (p,q)15={(2,0),(0,2)} (25)

[0125] The coordinate offset (p, q) of the G pixel used for interpolation in the diagonal direction of the lower right corner point is shown in formula (26):

[0126] (p,q)12={(0,-1),(-1,0),(2,1),(1,2)} (26)

[0127] The pixel value G of pixel G in the interpolation direction is calculated according to formula (27). i+p,j+q The W pixel value W in the half-resolution W image i,j The difference is used as the color difference value of the G pixel.

[0128]

[0129] The weight of each pixel involved in interpolation is its pixel value W. i+p,j+q and the target pixel value W i+m,j+b The reciprocal of the difference between them, weight i+p,j+q The calculation formula is shown in formula (28):

[0130]

[0131] According to formula (28), the weights for interpolating the B pixel point with G pixels and the R pixel point with G pixels are obtained respectively.

[0132] Based on the color difference of G pixels and the weight W of the target pixel i+m,j+n Perform weighted interpolation to obtain the G pixel value of the target pixel. i+m,j+n The calculation method is shown in formula (29):

[0133]

[0134] in,

[0135] G pixel value G i+m,j+n This generates a 100% G-pixel half-resolution image.

[0136] In another specific implementation of the embodiments of this application, see [link to relevant documentation]. Figure 20 Step S33 includes:

[0137] Step S331: Calculate the difference between the pixel value of the first pixel related to the second pixel in the interpolation direction and the W pixel value in the half-resolution W image as the first color difference value of the first pixel, and calculate the weight of the first pixel related to the second pixel in the interpolation direction.

[0138] Step S332: Based on the first color difference value and weight of the first pixel, perform weighted interpolation on the second pixel to generate the 50% half-resolution image of the first pixel.

[0139] In another specific implementation of the embodiments of this application, see [link to relevant documentation]. Figure 21 Step S34 includes:

[0140] Step S341: Calculate the difference between the pixel value of the second pixel point related to the first pixel point in the interpolation direction and the W pixel value in the half-resolution W image as the first color difference value of the second pixel point; and calculate the weight of the second pixel point related to the first pixel point in the interpolation direction.

[0141] Step S342: Based on the first color difference value and weight of the second pixel, perform weighted interpolation on the first pixel to generate the 50% second pixel half-resolution image.

[0142] Specifically, this application embodiment uses the example of the first pixel being a B pixel, the second pixel being an R pixel, and the third pixel being a G pixel. However, this application is not limited to this; the first pixel, the second pixel, and the third pixel can be R, G, and B pixels respectively, as needed.

[0143] Since generating a 50% B-pixel half-resolution image by interpolating B-pixels based on the gradient values ​​in each direction and the half-resolution W image is based on the same principle as generating a 50% R-pixel half-resolution image by interpolating B-pixels based on the gradient values ​​in each direction and the half-resolution W image, this application will only describe the generation of a 50% R-pixel half-resolution image by interpolating B-pixels based on the gradient values ​​in each direction and the half-resolution W image.

[0144] The pixels involved in interpolation at the top left, top right, bottom left, and bottom right corners in the horizontal, vertical, diagonal, and diagonal directions are respectively as follows: Figure 22 As shown in (A) to 22(P). That is, the situation of the first pixel related to the second pixel in the interpolation direction in step S331 is as follows. Figure 22 As shown in (A) to 22(P), the situation of the second pixel related to the first pixel in the interpolation direction in step S341 is as follows: Figure 22As shown in (A) to 22(P).

[0145] Figure 22 In (A) to 22(P), the horizontal coordinate axis is j and the vertical coordinate axis is i. The coordinate offset is marked with numbers, and the pixel marked with a circle is the target pixel. (p,q) is the coordinate offset of each corner point.

[0146] In step S331, the B pixels used for horizontal interpolation of the R pixels located at the top left and bottom left corners are respectively as follows: Figure 22 As shown in (A) and 22(D), the coordinate offset (p,q)17 of pixel B is as shown in formula (30):

[0147] (p,q)17={(-1,-2),(-1,-1),(-1,2),(2,-2),(2,-1),(2,2)} (30)

[0148] In step S341, the coordinate offset of the R pixel used for horizontal interpolation of the B pixel located at the upper left corner and the lower left corner is also as shown in formula (30).

[0149] In step S331, the B pixels used for horizontal interpolation of the R pixels located at the upper right corner and the lower right corner are respectively as follows: Figure 22 As shown in (E) and 22(M), the coordinate offset (p,q)18 of pixel B is as shown in formula (31):

[0150] (p,q)18={(-1,-1),(-1,2),(-1,3),(2,-1),(2,2),(2,3)} (31)

[0151] In step S341, the coordinate offset of the R pixel used for horizontal interpolation of the B pixel located at the upper right corner and the lower right corner is also as shown in formula (31).

[0152] In step S331, the B pixels used for vertical interpolation of the R pixels located at the top left and top right corners are respectively as follows: Figure 22 As shown in (B) and 22(F), the vertical coordinate offset (p,q)19 of pixel B is as shown in formula (32):

[0153] (p,q)19={(-2,-1),(-1,-1),(2,-1),(-2,2),(-1,2),(2,2)} (32)

[0154] In step S341, the coordinate offset of the R pixel used for vertical interpolation of the B pixel located at the upper left and upper right corners is also shown in formula (32).

[0155] In step S331, the B pixels used for vertical interpolation of the R pixels located at the lower left and lower right corners are respectively as follows: Figure 22 As shown in (J) and 22(N), the vertical coordinate offset (p,q)20 of pixel B is as shown in formula (33):

[0156] (p,q)20={(-1,-1),(2,-1),(3,-1),(-1,2),(2,2),(3,2)} (33)

[0157] In step S341, the coordinate offset of the R pixel used for vertical interpolation of the B pixel located at the lower left and lower right corners is also shown in formula (33).

[0158] In step S331, the B pixel used for diagonal interpolation of the R pixel located at the top left corner is as follows: Figure 22 As shown in (C), the coordinate offset (p,q)21 of pixel B is as shown in formula (34):

[0159] (p,q)21={(2,-2),(-2,2)} (34)

[0160] In step S341, the coordinate offset of the R pixel used for diagonal interpolation of the B pixel located at the upper left corner is also shown in formula (34).

[0161] In step S331, the B pixel used for diagonal interpolation of the R pixel located at the upper right corner is as follows: Figure 22 As shown in (G), the coordinate offset (p,q)22 of pixel B is as shown in formula (35):

[0162] (p,q)22={(2,-1),(-1,2),(-2,3)} (35)

[0163] In step S341, the coordinate offset of the R pixel used for diagonal interpolation of the B pixel located at the upper right corner is also shown in formula (35).

[0164] In step S331, the B pixel used for diagonal interpolation of the R pixel located at the lower left corner is as follows: Figure 22 As shown in (K), the coordinate offset (p,q)23 of pixel B is as shown in formula (36):

[0165] (p,q)23={(3,-2),(2,-1),(-1,2)} (36)

[0166] In step S341, the coordinate offset of the R pixel used for diagonal interpolation of the B pixel located at the lower left corner is also shown in formula (36).

[0167] In step S331, the B pixel used for diagonal interpolation of the R pixel located at the lower right corner is as follows: Figure 22 As shown in (O), the coordinate offset (p,q)24 of pixel B is as shown in formula (37):

[0168] (p,q)24={(3,-1),(-1,3)} (37)

[0169] In step S341, the coordinate offset of the R pixel used for diagonal interpolation of the B pixel located at the lower right corner is also shown in formula (37).

[0170] In step S331, the B pixel used for diagonal interpolation of the R pixel located at the top left corner is as follows: Figure 22 As shown in (D), the coordinate offset (p,q)25 of pixel B in the diagonal direction is as shown in formula (38):

[0171] (p,q)25={(-2,-2),(-1,-1),(2,2)} (38)

[0172] In step S341, the coordinate offset of the R pixel used for diagonal interpolation of the B pixel located at the upper left corner is also as shown in formula (38).

[0173] In step S331, the B pixel used for diagonal interpolation of the R pixel located at the upper right corner is as follows: Figure 22 As shown in (H), the coordinate offset (p,q)26 of pixel B is as shown in formula (39):

[0174] (p,q)26={(-2,-1),(2,3)} (39)

[0175] In step S341, the coordinate offset of the R pixel used for diagonal interpolation of the B pixel located at the upper right corner is also as shown in formula (39).

[0176] In step S331, the B pixel used for diagonal interpolation of the R pixel located at the lower left corner is as follows: Figure 22 As shown in (L), the coordinate offset (p,q)27 of pixel B is as shown in formula (40):

[0177] (p,q)27={(-1,-2),(3,2)} (40)

[0178] In step S341, the coordinate offset of the R pixel used for diagonal interpolation of the B pixel located at the lower left corner is also as shown in formula (40).

[0179] In step S331, the B pixel used for diagonal interpolation of the R pixel located at the lower right corner is as follows: Figure 22 As shown in (P), the coordinate offset (p,q)28 of pixel B is as shown in formula (41):

[0180] (p,q)28={(-1,-1),(2,2),(3,3)} (41)

[0181] In step S341, the coordinate offset of the R pixel used for diagonal interpolation of the B pixel located at the lower right corner is also as shown in formula (41).

[0182] The pixel value R1 of pixel R in the interpolation direction is calculated according to formula (42). i+p,j+q The W pixel value W in the half-resolution W image i+p,j+q The difference is used as the first color difference value of the R pixel.

[0183]

[0184] According to formula (28), the weights for interpolating the B pixel point into the R pixel point are obtained respectively. i+p,j+q .

[0185] Based on the first color difference value of pixel R Weight of the target pixel i+p,j+q Perform weighted interpolation to obtain the R pixel value R1 of the target pixel. i+m,j+n The calculation method is shown in formula (43):

[0186]

[0187] in,

[0188] R pixel value R1 i+m,j+n This generates a half-resolution image with 50% of the R pixels.

[0189] In another specific implementation of the embodiments of this application, see [link to relevant documentation]. Figure 23 Step S35 includes:

[0190] Step S351: Calculate the difference between the pixel value of the first pixel related to the third pixel in the interpolation direction and the pixel value of W in the half-resolution W image as the second color difference value of the first pixel; and calculate the weight of the first pixel related to the third pixel in the interpolation direction.

[0191] Step S352: Based on the second color difference value and weight of the first pixel, perform weighted interpolation on the third pixel to generate a 100% half-resolution image of the first pixel.

[0192] In another specific implementation of the embodiments of this application, see [link to relevant documentation]. Figure 24 Step S36 includes:

[0193] Step S361: Calculate the difference between the pixel value of the second pixel point related to the third pixel point in the interpolation direction and the W pixel value in the half-resolution W image as the second color difference value of the second pixel point; and calculate the weight of the second pixel point related to the third pixel point in the interpolation direction.

[0194] Step S362: Based on the second color difference value and weight of the second pixel, perform weighted interpolation on the third pixel to generate a 100% second pixel half-resolution image.

[0195] Specifically, this application embodiment uses the example of the first pixel being a B pixel, the second pixel being an R pixel, and the third pixel being a G pixel. However, this application is not limited to this; the first pixel, the second pixel, and the third pixel can be R, G, and B pixels respectively, as needed.

[0196] The principle behind generating a 100% B-pixel half-resolution image by interpolating the 50% B-pixel half-resolution image at the G-pixel point using the half-resolution W image and the gradient values ​​in each direction is the same as generating a 100% R-pixel half-resolution image by interpolating the 50% R-pixel half-resolution image at the G-pixel point using the half-resolution W image and the gradient values ​​in each direction. This application will only describe the generation of a 100% R-pixel half-resolution image by interpolating the 50% R-pixel half-resolution image at the G-pixel point using the half-resolution W image and the gradient values ​​in each direction.

[0197] The pixels involved in interpolation at the top left, top right, bottom left, and bottom right corners in the horizontal, vertical, diagonal, and oblique diagonal directions are consistent with those described in the interpolation of G pixels at B and R pixels, i.e., as... Figures 19(A) to 19(P) As shown.

[0198] Calculate the pixel value G2 of the G pixel in the interpolation direction for the 50% B pixel half-resolution image according to formula (44). i+p,j+q The W pixel value W in the half-resolution W image i,j The difference is used as the second color difference value of the G pixel.

[0199]

[0200] According to formula (28), the weights for interpolating the B pixel point into the R pixel point are obtained respectively. i+p,j+q .

[0201] It can be based on the second color difference value of pixel R. Weight of the target pixel i+p,j+q Perform weighted interpolation to obtain the R pixel value R2 of the target pixel. i+m,j+n The calculation method is shown in formula (45):

[0202]

[0203] in,

[0204] R pixel value R i+m,j+n This generates a half-resolution image with 100% R pixels.

[0205] This application embodiment utilizes a half-resolution W image to calculate gradients in each direction, calculates the color difference between each pixel in the R, G, and B regions and the W pixel in the half-resolution W image, as well as the weights of the weighted interpolation of each pixel, which can further suppress artifacts such as false colors and zipper effects.

[0206] In another specific implementation of the embodiments of this application, step S4 includes:

[0207] The half-resolution R, G, and B images are subjected to bilinear interpolation downsampling and then fused to obtain a full-resolution RGB image.

[0208] For the methods described above, please refer to [link / reference]. Figure 25 This application also provides an image processing apparatus, including:

[0209] The W image generation module 251 is used to perform W interpolation on the HexRGBW data obtained from the HexRGBW configuration color filter array to generate a full-resolution W image.

[0210] The first sampling module 252 is used to perform a first sampling process based on the full-resolution W image and the HexRGBW data to generate a half-resolution W image and a half-resolution RGB image.

[0211] The RGB interpolation module 253 is used to perform RGB interpolation processing based on the half-resolution W image and the half-resolution RGB to generate half-resolution R image, G image and B image.

[0212] The second sampling module 254 is used to perform second sampling processing on the half-resolution R image, G image, and B image to generate a full-resolution RGB image.

[0213] This application also provides a computer program product tangibly stored on a readable medium of a controller, and computer-executable instructions that, when executed, cause at least one processor to perform the method according to any one of the above-described schemes.

[0214] This application also provides a computer-readable medium storing computer-executable instructions thereon, which, when executed, cause at least one processor to perform the method according to any one of the above-described schemes.

[0215] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0216] The methods described in the embodiments of this application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code downloaded over a network that is originally stored in a remote recording medium or a non-transitory machine-readable medium and will be stored in a local recording medium. Thus, the methods described herein can be processed by software stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code that, when accessed and executed by the computer, processor, or hardware, implements the checksum generation method described herein. Furthermore, when a general-purpose computer accesses code used to implement the checksum generation method shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the checksum generation method shown herein.

[0217] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0218] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. An image processing method, characterized in that, include: The HexRGBW data obtained from the HexRGBW configuration color filter array is interpolated by W to generate a full-resolution W image. The first sampling process is performed on the full-resolution W image and the HexRGBW data to generate a half-resolution W image and a half-resolution RGB image, respectively. Based on the half-resolution W image, the half-resolution RGB image is subjected to RGB interpolation to generate half-resolution R image, G image, and B image; as well as A second sampling process is performed on the half-resolution R image, G image, and B image to generate a full-resolution RGB image; The step of performing RGB interpolation processing on the half-resolution RGB image based on the half-resolution W image to generate half-resolution R, G, and B images includes: Gradient calculation is performed on the half-resolution W image to obtain the gradient values ​​of the half-resolution W image in multiple directions, including: horizontal, vertical, diagonal, and diagonal directions. Based on the gradient values ​​in the multiple directions and the half-resolution W image, the third pixel value is interpolated for the first and second pixels in the R, G, and B pixels of the half-resolution RGB image to generate a 100% third pixel half-resolution image. Based on the gradient values ​​in the multiple directions, the second pixel value of the R, G, and B pixels in the half-resolution RGB image is interpolated using the half-resolution W image to generate a 50% first pixel half-resolution image. Based on the gradient values ​​in the multiple directions, the second pixel value is interpolated for the first pixel in the R, G, and B pixels of the half-resolution RGB image of the half-resolution W image to generate a 50% second pixel half-resolution image. Based on the gradient values ​​in the multiple directions, the half-resolution W image is used to interpolate the first pixel value at the third pixel point of the 50% first pixel half-resolution image, generating a 100% first pixel half-resolution image; and Based on the gradient values ​​in the multiple directions and the half-resolution W image, the second pixel value of the 50% second pixel half-resolution image is interpolated at the third pixel to generate a 100% second pixel half-resolution image. The step of performing W interpolation on the HexRGBW data obtained from the HexRGBW configuration color filter array to generate a full-resolution W image includes: Obtain the horizontal and vertical gradients of the W value in the HexRGBW data at non-white pixels; If the horizontal gradient is less than a first threshold and the vertical gradient is less than a second threshold, the W value of the non-white pixel is determined by isotropic interpolation. If the absolute value of the difference between the horizontal gradient and the vertical gradient is less than the third threshold, the W value of the non-white pixel is determined by isotropic interpolation. If the absolute value of the difference between the horizontal gradient and the vertical gradient is greater than the third threshold, and the horizontal gradient is less than the vertical gradient, the W value of the non-white pixel is determined by horizontal interpolation; if the absolute value of the difference between the horizontal gradient and the vertical gradient is greater than the third threshold, and the horizontal gradient is greater than the vertical gradient, the W value of the non-white pixel is determined by vertical interpolation.

2. The image processing method according to claim 1, characterized in that, The first sampling process is a downsampling process. The step of performing the first sampling process based on the full-resolution W image and the HexRGBW data to generate a half-resolution W image and a half-resolution RGB image includes: The half-resolution W image is obtained by arithmetically averaging the W values ​​in the diagonal directions of the full-resolution W image; and The half-resolution RGB image is obtained by performing an arithmetic average of the R, G, and B values ​​in the diagonal direction of the HexRGBW data.

3. The image processing method according to claim 1, characterized in that, The step of performing gradient calculation based on the half-resolution W image to obtain the gradient values ​​of the half-resolution W image in multiple directions includes: The gradient value in the horizontal direction is obtained based on the absolute value of the difference between multiple pairs of horizontal pixels related to the pixel to be interpolated in the half-resolution W image. The gradient value in the vertical direction is obtained based on the absolute value of the difference between multiple pairs of vertically related pixels in the half-resolution W image. The gradient value in the diagonal direction is obtained based on the absolute value of the difference between multiple pairs of diagonal pixels related to the pixel to be interpolated in the half-resolution W image. The gradient value in the diagonal direction is obtained by using the absolute value of the difference between multiple pairs of diagonal pixels related to the pixel to be interpolated in the half-resolution W image.

4. The image processing method according to claim 3, characterized in that, The step of interpolating the third pixel value of the first and second pixels among the R, G, and B pixels of the half-resolution RGB image based on the gradient values ​​in the multiple directions and the half-resolution W image to generate a 100% third pixel half-resolution image includes: Based on the gradient values ​​in the multiple directions, the direction corresponding to the minimum gradient is determined as the interpolation direction; The difference between the pixel value of the third pixel related to the first pixel and the second pixel in the interpolation direction and the W pixel value in the half-resolution W image is calculated as the color difference value of the third pixel, and the weights of the third pixels related to the first pixel and the second pixel in the interpolation direction are calculated; and Based on the color difference value and weight of the third pixel, weighted interpolation is performed on the first pixel and the second pixel to generate the 100% half-resolution image of the third pixel.

5. The image processing method according to claim 4, characterized in that, The step of interpolating the first pixel value of the second pixel in the R, G, and B pixels of the half-resolution RGB image based on the gradient values ​​in the multiple directions and the half-resolution W image to generate a 50% half-resolution image of the first pixel includes: The difference between the pixel value of the first pixel related to the second pixel in the interpolation direction and the pixel value of W in the half-resolution W image is calculated as the first color difference value of the first pixel, and the weight of the first pixel related to the second pixel in the interpolation direction is calculated; and Based on the first color difference value and weight of the first pixel, the second pixel is weighted and interpolated to generate the 50% half-resolution image of the first pixel.

6. The image processing method according to claim 5, characterized in that, The step of interpolating the first pixel of the R, G, and B pixels in the half-resolution RGB image based on the gradient values ​​in the multiple directions and the half-resolution W image to generate a 50% second pixel half-resolution image includes: The difference between the pixel value of the second pixel related to the first pixel in the interpolation direction and the pixel value of W in the half-resolution W image is calculated as the first color difference value of the second pixel, and the weight of the second pixel related to the first pixel in the interpolation direction is calculated; and Based on the first color difference value and weight of the second pixel, the first pixel is weighted and interpolated to generate the 50% second pixel half-resolution image.

7. The image processing method according to claim 6, characterized in that, The step of interpolating the first pixel value at the third pixel point of the 50% first pixel half-resolution image based on the gradient values ​​in the multiple directions and the half-resolution W image to generate a 100% first pixel half-resolution image includes: The difference between the pixel value of the third pixel point and the pixel value of W in the half-resolution W image along the interpolation direction is calculated as the second color difference value of the first pixel point, and the weight of the third pixel point relative to the first pixel point along the interpolation direction is calculated; and Based on the second color difference value and weight of the first pixel, the third pixel is weighted and interpolated to generate a half-resolution image of 100% of the first pixel.

8. The image processing method according to claim 7, characterized in that, The step of interpolating the second pixel value at the third pixel point of the 50% second pixel half-resolution image based on the gradient values ​​in the multiple directions and the half-resolution W image to generate a 100% second pixel half-resolution image includes: The difference between the pixel value of the second pixel point related to the third pixel point in the interpolation direction and the W pixel value in the half-resolution W image is calculated as the second color difference value of the second pixel point; Calculate the weight of the second pixel relative to the third pixel in the interpolation direction; Based on the second color difference value and weight of the second pixel, the third pixel is weighted and interpolated to generate a half-resolution image of 100% of the second pixel.

9. The image processing method according to claim 1, characterized in that, The second sampling process is an upsampling process. The step of performing the second sampling process based on the half-resolution R image, G image, and B image to generate a full-resolution RGB image includes: The half-resolution R, G, and B images are subjected to bilinear interpolation and fused to obtain a full-resolution RGB image.

10. An image processing apparatus, characterized in that, include: The W image generation module is used to perform W interpolation on the HexRGBW data obtained by the HexRGBW configuration color filter array to generate a full-resolution W image. The first sampling module is used to perform a first sampling process based on the full-resolution W image and the HexRGBW data to generate a half-resolution W image and a half-resolution RGB image. The RGB interpolation module is used to perform RGB interpolation processing on the half-resolution RGB image based on the half-resolution W image to generate half-resolution R, G, and B images. as well as The second sampling module is used to perform second sampling processing on the half-resolution R image, G image, and B image to generate a full-resolution RGB image; The RGB interpolation module is specifically used for: Gradient calculation is performed on the half-resolution W image to obtain the gradient values ​​of the half-resolution W image in multiple directions, including: horizontal, vertical, diagonal, and diagonal directions. Based on the gradient values ​​in the multiple directions and the half-resolution W image, the third pixel value is interpolated for the first and second pixels in the R, G, and B pixels of the half-resolution RGB image to generate a 100% third pixel half-resolution image. Based on the gradient values ​​in the multiple directions, the second pixel value of the R, G, and B pixels in the half-resolution RGB image is interpolated using the half-resolution W image to generate a 50% first pixel half-resolution image. Based on the gradient values ​​in the multiple directions, the second pixel value is interpolated for the first pixel in the R, G, and B pixels of the half-resolution RGB image of the half-resolution W image to generate a 50% second pixel half-resolution image. Based on the gradient values ​​in the multiple directions, the half-resolution W image is used to interpolate the first pixel value at the third pixel point of the 50% first pixel half-resolution image, generating a 100% first pixel half-resolution image; and Based on the gradient values ​​in the multiple directions and the half-resolution W image, the second pixel value of the 50% second pixel half-resolution image is interpolated at the third pixel to generate a 100% second pixel half-resolution image. The step of performing W interpolation on the HexRGBW data obtained from the HexRGBW configuration color filter array to generate a full-resolution W image includes: Obtain the horizontal and vertical gradients of the W value in the HexRGBW data at non-white pixels; If the horizontal gradient is less than a first threshold and the vertical gradient is less than a second threshold, the W value of the non-white pixel is determined by isotropic interpolation. If the absolute value of the difference between the horizontal gradient and the vertical gradient is less than the third threshold, the W value of the non-white pixel is determined by isotropic interpolation. If the absolute value of the difference between the horizontal gradient and the vertical gradient is greater than the third threshold, and the horizontal gradient is less than the vertical gradient, the W value of the non-white pixel is determined by horizontal interpolation; if the absolute value of the difference between the horizontal gradient and the vertical gradient is greater than the third threshold, and the horizontal gradient is greater than the vertical gradient, the W value of the non-white pixel is determined by vertical interpolation.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a computer, causes the computer to perform the image processing method as described in any one of claims 1 to 9.

Images