Interpolation method and device for RAW image in tetrabayer format and electronic equipment

Abstract

The invention provides an interpolation method and device for a RAW image in a tetrabayer format and electronic equipment. The interpolation method comprises the following steps: acquiring the RAW image in the tetrabayer format; gradient information of the current pixel point is calculated; calculating a weight factor of the current pixel point according to the gradient information; and calculating a G channel value, a G-R color difference value and a G-B color difference value of the current pixel point by using the weight factor so as to obtain an RGB channel value of the current pixel point and output an RGB image. Gradient information of a current pixel point is calculated, and then a weight factor is calculated, so that edge information of the image can be obtained; the RGB channel value of the current pixel point is calculated through the weight factor, an image conversion result with small noise in a flat area and clear details in an edge area can be obtained, an existing demosaicing algorithm is not needed, conversion from the RAW image in the tetrabayer format to the RGB image is achieved, the image quality is guaranteed, and the image conversion efficiency is improved. The problem of how to convert the RAW image in the tetrabayer format into the high-quality color image is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an interpolation method and device for a four-Bayer format RAW image, and electronic equipment. Background Art

[0002] Image sensors are devices that convert optical images into electronic signals. They are widely used in digital cameras and other electronic optical devices. There is a huge difference in the sensitivity of image sensors in environments with sufficient light and insufficient light. For example, when there is insufficient light at night, the image sensor is prone to a decrease in signal-to-noise ratio, which makes the denoising effect in the later image signal processing process unsatisfactory.

[0003] In order to improve the imaging quality in dark environments, a new type of image sensor is proposed, whose RAW images are arranged in a quad-Bayer format. Since the arrangement of pixels in the RAW images in the quad-Bayer format is equivalent to splitting each pixel in the existing ordinary Bayer format into four pixels of the same color, the signal-to-noise ratio can be improved by merging effective pixels to achieve better low-light environment performance.

[0004] Since the format of the four-Bayer format image is different from the existing ordinary Bayer format, the existing image signal processing algorithm cannot be directly used for subsequent processing. At present, the four-Bayer format image is usually converted into the existing ordinary Bayer format image using a demosaicing algorithm, and then subsequent image signal processing is performed to output a color image. However, in this existing method, since the format of the four-Bayer format RAW image is different from the existing ordinary Bayer format, when passing through the demosaicing algorithm, the image quality will be lost to a certain extent, affecting the perception of the final output color image. Summary of the invention

[0005] The object of the present invention is to provide a method and device for interpolating a four-Bayer format RAW image, and an electronic device, so as to solve the problem of how to convert the four-Bayer format RAW image into a high-quality color image.

[0006] In order to solve the above technical problems, the present invention provides a method for interpolating a RAW image in a four-Bayer format, comprising: Get quad-Bayer format RAW images; Calculate the gradient information of the current pixel; According to the gradient information, calculate the weight factor of the current pixel; Using the weight factor, calculate the G channel value, GR color difference value and GB color difference value of the current pixel; Using the G channel value, GR color difference value and GB color difference value, the G channel value, R channel value and B channel value of the current pixel are obtained; Combine the G channel value, R channel value, and B channel value of the current pixel to output an RGB image.

[0007] Optionally, in the interpolation method of the four-Bayer format RAW image, the method of calculating the gradient information of the current pixel point includes: The horizontal gradient of the current pixel is calculated using the four-Bayer pixel where the current pixel is located. , vertical gradient , diagonal gradient and the anti-diagonal gradient , expressed as:

[0008]

[0009]

[0010]

[0011] in, Represents the pixel value of the pixel located in the upper left corner of the four-Bayer pixel, Represents the pixel value of the pixel located in the upper right corner of the four-Bayer pixel, Represents the pixel value of the pixel at the lower left corner of the four-Bayer pixel, Indicates the pixel value of the pixel located at the lower right corner in the four-Bayer pixel; the gradient information of the four pixels in the same four-Bayer pixel is consistent.

[0012] Optionally, in the interpolation method of the four-Bayer format RAW image, the method of calculating the weight factor of the current pixel point according to the gradient information includes: According to the horizontal gradient of the current pixel , vertical gradient , diagonal gradient and the anti-diagonal gradient The minimum value among them confirms the direction information of the current pixel; According to the horizontal gradient of the current pixel , vertical gradient , diagonal gradient and the anti-diagonal gradient The maximum value among them confirms the edge strength of the current pixel; Calculate the weight factor of the current pixel based on the direction information and edge strength of the current pixel.

[0013] Optionally, in the interpolation method of the four-Bayer format RAW image, the method of calculating the weight factor of the current pixel point according to the direction information and edge strength of the current pixel point includes: Count the direction information of all four Bayer pixels to obtain a direction matrix; The number of pixels in the direction matrix that are consistent with the direction information of the current pixel is counted to obtain the direction credibility of the current pixel. According to the direction information of the current pixel, the convolution kernel of the current pixel is determined; Using the convolution kernel, edge strength and direction information, calculate the intermediate matrix of the current pixel; The weight factor of the current pixel is calculated using the intermediate matrix and the distance vector between the current pixel and the surrounding pixels.

[0014] Optionally, in the interpolation method of the four-Bayer format RAW image, the method of determining the convolution kernel of the current pixel point according to the direction information of the current pixel point includes: Construct an anisotropic Gaussian kernel where: If the horizontal gradient of the current pixel , vertical gradient , diagonal gradient and the anti-diagonal gradient The minimum value in is the horizontal gradient, so the direction information of the current pixel is the horizontal direction, and the convolution kernel of the anisotropic Gaussian kernel , ; If the horizontal gradient of the current pixel , vertical gradient , diagonal gradient and the anti-diagonal gradient The minimum value in is the vertical gradient, so the direction information of the current pixel is the vertical direction, and the convolution kernel of the anisotropic Gaussian kernel is , ; If the horizontal gradient of the current pixel , vertical gradient , diagonal gradient and the anti-diagonal gradient The minimum value in is the diagonal gradient, so the direction information of the current pixel is the diagonal direction, and the convolution kernel of the anisotropic Gaussian kernel , ; If the horizontal gradient of the current pixel , vertical gradient , diagonal gradient and the anti-diagonal gradient The minimum value in is the anti-diagonal gradient, so the direction information of the current pixel is the anti-diagonal direction, and the convolution kernel of the anisotropic Gaussian kernel , .

[0015] Optionally, in the interpolation method of the four-Bayer format RAW image, the intermediate matrix O of the current pixel is calculated by using the convolution kernel, edge strength and direction information, which is expressed as:

[0016] in, and Represent the vectors of the major axis and minor axis of the elliptical convolution kernel respectively; and Respectively represent and A vector with a perpendicular direction; represents the edge strength, ; is an adjustable parameter; m is the direction credibility, It is a set of arrays, confirmed by direction credibility The value of .

[0017] Optionally, in the interpolation method of the four-Bayer format RAW image, the weight factor of the current pixel is calculated by using the intermediate matrix and the distance vector between the current pixel and the surrounding pixels. , expressed as:

[0018] in, Represents the distance vector between the current pixel and the surrounding pixels, and O represents the intermediate matrix.

[0019] Optionally, in the interpolation method of the four-Bayer format RAW image, the method of calculating the G channel value, GR color difference value and GB color difference value of the current pixel point by using the weight factor includes: If the current pixel is an R pixel or a B pixel, the weight factor is used to perform weighted average calculation on the G pixels around it to obtain the G channel value of the current pixel; If the current pixel is an R pixel or a B pixel, the color difference is calculated using the pixel value of the current pixel and the G channel value obtained by interpolation to obtain the GR color difference value of the R pixel and the GB color difference value of the B pixel; If the current pixel is an R pixel, the weight factor is used to perform weighted average calculation on the GB color difference values ​​of the B pixels around the current pixel to obtain the GB color difference value of the R pixel; if the current pixel is a B pixel, the weight factor is used to perform weighted average calculation on the GR color difference values ​​of the R pixels around the current pixel to obtain the GR color difference value of the B pixel; if the current pixel is a G pixel, the weight factor is used to perform weighted average calculation on the GB color difference values ​​of the B pixels and the GR color difference values ​​of the R pixels around the current pixel to obtain the GB color difference value and GR color difference value of the G pixel.

[0020] Optionally, in the interpolation method of the four-Bayer format RAW image, the method of obtaining the G channel value, R channel value and B channel value of the current pixel point by using the G channel value, GR color difference value and GB color difference value includes: If the current pixel is a G pixel, the pixel value of the current pixel and the calculated GB color difference value and GR color difference value are subtracted to obtain the B channel value and the R channel value; If the current pixel is an R pixel, the G channel value calculated for the current pixel and the GB color difference value are subtracted to obtain the B channel value; If the current pixel is a B pixel, a difference calculation is performed on the G channel value and the GR color difference value calculated for the current pixel to obtain an R channel value.

[0021] In order to solve the above technical problems, the present invention further provides an interpolation device for a RAW image in a four-Bayer format, which is used to implement the interpolation method for a RAW image in a four-Bayer format as described in any one of the above items. The interpolation device for a RAW image in a four-Bayer format includes: An image acquisition module, used for acquiring a four-Bayer format RAW image; Gradient calculation module, used to calculate the gradient information of the current pixel; The weight factor calculation module is used to calculate the weight factor of the current pixel point according to the gradient information; The color difference calculation module is used to calculate the G channel value, GR color difference value and GB color difference value of the current pixel point using the weight factor; The full channel calculation module is used to obtain the G channel value, R channel value and B channel value of the current pixel point by using the G channel value, GR color difference value and GB color difference value; The image output module is used to combine the G channel value, R channel value and B channel value of the current pixel point to output an RGB image.

[0022] In order to solve the above technical problems, the present invention also provides an electronic device, comprising a memory, a processor, and an executable program stored in the memory and capable of being run by the processor; when the processor runs the executable program, the interpolation method of the four-Bayer format RAW image as described in any one of the above items is executed.

[0023] The interpolation method and device of the four-Bayer format RAW image provided by the present invention, and the electronic device, include: obtaining the four-Bayer format RAW image; calculating the gradient information of the current pixel; calculating the weight factor of the current pixel according to the gradient information; calculating the G channel value, GR color difference value and GB color difference value of the current pixel using the weight factor; obtaining the G channel value, R channel value and B channel value of the current pixel using the G channel value, GR color difference value and GB color difference value; combining the G channel value, R channel value and B channel value of the current pixel to output an RGB image. By calculating the gradient information of the current pixel and then calculating the weight factor, the edge information of the image can be obtained; by calculating the RGB channel value of the current pixel using the weight factor, the image conversion result with low noise in the flat area and clear details in the edge area can be obtained, which not only does not require the existing demosaicing algorithm, but also realizes the conversion of the four-Bayer format RAW image to the RGB image, and also ensures the image quality, solving the problem of how to convert the four-Bayer format RAW image into a high-quality color image. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A flowchart of the interpolation method of a four-Bayer format RAW image provided in this embodiment; Figure 2 A schematic diagram of the pixel arrangement structure of a RAW image in a quad-Bayer format provided in this embodiment; Figure 3 A schematic diagram showing the gradient direction and calculation of the current pixel point provided in this embodiment; Figure 4 A schematic diagram of the arrangement structure of four Bayer pixels in a 6×6 first cache window provided in this embodiment; Figure 5 A schematic diagram of the arrangement structure of pixels in a 5×5 second cache window with a central pixel being a B pixel provided in this embodiment; Figure 6 A schematic diagram of calculating the weight factor corresponding to a 5×5 second cache window whose central pixel is a B pixel provided in this embodiment; Figure 7 A schematic diagram of color difference arrangement corresponding to a 5×5 second cache window with a G pixel as the central pixel provided in this embodiment; Figure 8 A schematic diagram of color difference arrangement corresponding to a 5×5 second cache window with a central pixel being a B pixel provided in this embodiment; Fig. 9 This is a structural block diagram of the interpolation device for a quad-Bayer format RAW image provided in this embodiment. DETAILED DESCRIPTION

[0025] The following is a further detailed description of the interpolation method and device for the RAW image in the four-Bayer format and the electronic device proposed by the present invention in combination with the accompanying drawings and specific embodiments. It should be noted that the accompanying drawings are all in a very simplified form and use non-precise proportions, which are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention. In addition, the structure shown in the accompanying drawings is often a part of the actual structure. In particular, the emphasis of each accompanying drawing is different, and sometimes different proportions are used.

[0026] It should be noted that the terms "first", "second", etc. in the specification, claims and drawings of the present invention are used to distinguish similar objects in order to describe the embodiments of the present invention, rather than to describe a specific order or sequence. It should be understood that the structures used in this way can be interchanged under appropriate circumstances. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] This embodiment provides a method for interpolating a RAW image in a four-Bayer format. Figure 1 As shown, including: S1, acquiring a four-Bayer format RAW image; S2, calculate the gradient information of the current pixel; S3, calculate the weight factor of the current pixel according to the gradient information; S4, using the weight factor, calculating the G channel value, GR color difference value and GB color difference value of the current pixel; S5, using the G channel value, the GR color difference value and the GB color difference value, obtain the G channel value, the R channel value and the B channel value of the current pixel; S6, combining the G channel value, the R channel value, and the B channel value of the current pixel to output an RGB image.

[0028] The interpolation method for a four-Bayer format RAW image provided in this embodiment can obtain the edge information of the image by calculating the gradient information of the current pixel point and then calculating the weight factor; the RGB channel value of the current pixel point is calculated by the weight factor, and an image conversion result with small noise in a flat area and clear details in the edge area can be obtained. Not only does it not require the existing demosaicing algorithm, it also realizes the conversion of the four-Bayer format RAW image to the RGB image, but also ensures the image quality, solving the problem of how to convert the four-Bayer format RAW image into a high-quality color image.

[0029] Specifically, in this embodiment, in step S1, the method for obtaining a RAW image in a four-Bayer format may be to obtain an original RAW image in a four-Bayer format from various photographic and video equipment using a new type of image sensor.

[0030] Raw images are original image files obtained from image sensors of devices such as digital cameras, scanners, or film scanners. Raw images are not processed and therefore have internal colors with a wide color gamut, allowing for precise adjustments.

[0031] In a RAW image of the four-Bayer format, two adjacent G pixels in each row are separated by two adjacent R pixels or two B pixels, and there are only G pixels and R pixels or G pixels and B pixels in the same row; the pixel arrangement of every two rows is consistent, and the G pixels in the first two rows and the last two rows are staggered, and the G pixels in the first two rows and the last two rows are separated by R pixels and B pixels respectively; the arrangement of columns is consistent with that of rows. For example, Figure 2 As shown, when two adjacent G pixels in the i-th row are separated by two adjacent R pixels, the two adjacent G pixels in the i+1-th row are also separated by two adjacent R pixels, and the pixel arrangement of the i+1-th row is consistent with the pixel arrangement of the i-th row; the two adjacent G pixels in the i+2-th row are separated by two adjacent B pixels, and the two G pixels are staggered with the two G pixels in the i-th row; the two adjacent G pixels in the i+3-th row are also separated by two adjacent B pixels, and the pixel arrangement of the i+3-th row is consistent with the pixel arrangement of the i+2-th row.

[0032] Furthermore, in this embodiment, in step S2, the method for calculating the gradient information of the current pixel point includes: The horizontal gradient of the current pixel is calculated using the four-Bayer pixel where the current pixel is located. , vertical gradient , diagonal gradient and the anti-diagonal gradient , expressed as:

[0033]

[0034]

[0035]

[0036] in, Represents the pixel value of the pixel located in the upper left corner of the four-Bayer pixel, Represents the pixel value of the pixel located in the upper right corner of the four-Bayer pixel, Represents the pixel value of the pixel at the lower left corner of the four-Bayer pixel, Indicates the pixel value of the pixel located at the lower right corner of the four-Bayer pixel.

[0037] In a specific embodiment, if Figure 3 As shown, the current pixel is a B pixel (p1), and it and the three surrounding B pixels of the same color together form a four-Bayer B pixel. The four directional gradients of the current pixel can be obtained by using the above calculation method of gradient information, where the horizontal direction represents the 0 degree direction, the vertical direction represents the 90 degree direction, the diagonal direction represents the 45 degree direction, and the anti-diagonal direction represents the -45 degree direction.

[0038] It can be seen from the above calculation formula that the gradient information of the four pixels in the same four-Bayer pixel is consistent.

[0039] In this way, by calculating the gradient information using the four-Bayer pixels where the current pixel is located, it is possible to obtain the highest frequency spatial information while reducing the computational complexity and computing resources as much as possible, thereby obtaining more accurate image edge information, which helps to restore a clear image and avoid aliasing of interpolation results, thereby obtaining better image interpolation results.

[0040] Furthermore, in this embodiment, in step S3, the method of calculating the weight factor of the current pixel point according to the gradient information includes: S31, based on the horizontal gradient of the current pixel , vertical gradient , diagonal gradient and the anti-diagonal gradient The minimum value among them confirms the direction information of the current pixel.

[0041] Specifically, in this embodiment, if the minimum value of the horizontal gradient, vertical gradient, diagonal gradient and anti-diagonal gradient of the current pixel is the horizontal gradient, then the direction information of the current pixel is the horizontal direction (0 degrees); if the minimum value of the horizontal gradient, vertical gradient, diagonal gradient and anti-diagonal gradient of the current pixel is the vertical gradient, then the direction information of the current pixel is the vertical direction (90 degrees); if the minimum value of the horizontal gradient, vertical gradient, diagonal gradient and anti-diagonal gradient of the current pixel is the diagonal gradient, then the direction information of the current pixel is the diagonal direction (45 degrees); if the minimum value of the horizontal gradient, vertical gradient, diagonal gradient and anti-diagonal gradient of the current pixel is the anti-diagonal gradient, then the direction information of the current pixel is the anti-diagonal direction (-45 degrees).

[0042] Considering that four-Bayer pixels use less pixel data when calculating gradients, it is possible that the gradient calculation will be wrong due to factors such as noise, and the directions of the images in the neighborhood are similar. Therefore, in practical applications, the direction in the neighborhood can be calculated to see whether it is consistent with the direction of the current four-Bayer pixel to determine whether the gradient calculation is wrong, thereby ensuring the accuracy of the gradient calculation.

[0043] S32, based on the horizontal gradient of the current pixel , vertical gradient , diagonal gradient and the anti-diagonal gradient The maximum value among them confirms the edge strength of the current pixel, that is, the edge strength It can be expressed as:

[0044] S33, calculating a weight factor of the current pixel according to the direction information and edge strength of the current pixel.

[0045] Specifically, in this embodiment, first, the direction information of all four Bayer pixels is counted to obtain a direction matrix. Since the gradient information of the four pixels in the same four Bayer pixel is the same, the direction information of the four pixels is also the same. In this embodiment, only the direction information of the four Bayer pixels is counted.

[0046] In practical applications, a cache window can be set, such as a first cache window of size N×N, and the direction information of all four-Bayer pixels in the first cache window can be counted, where N=4x+2, x is a positive integer, which can be reasonably set according to actual needs. In this way, a × In a specific embodiment, a first cache window of size 6×6 is extracted, such as Figure 4 As shown, there are 9 four-Bayer pixels, and the direction information of each four-Bayer pixel is calculated separately to obtain a 3×3 direction matrix.

[0047] Then, the number of pixels in the direction matrix that are consistent with the direction information of the current pixel is counted to obtain the direction credibility of the current pixel. Taking the above 6×6 first cache window as an example, the number of four-Bayer pixels that are consistent with the direction of the current pixel can be 0~8. When the value is 0, it means that the directions of the other four-Bayer pixels in the first cache window are different from the direction of the current pixel; when the value is 8, it means that the directions of the other four-Bayer pixels in the first cache window are consistent with the direction of the current pixel. In other words, the closer the value of the direction credibility is to the maximum value, the better the direction consistency of each pixel in the current first cache window is, and the higher the direction credibility is.

[0048] Considering that the use of mean filtering can obtain the most ideal result when interpolating in the flat area of ​​the image, while interpolation along the edge direction is required to obtain better results when interpolating in the edge area of ​​the image, therefore, in this embodiment, an anisotropic Gaussian kernel is constructed so that both the edge area and the flat area of ​​the image can adaptively achieve the best interpolation effect.

[0049] Specifically, the convolution kernel of the current pixel is determined according to the direction information of the current pixel. In this embodiment, if the direction information of the current pixel is horizontal, the convolution kernel of the anisotropic Gaussian kernel is , ; If the direction information of the current pixel is vertical, the convolution kernel of the anisotropic Gaussian kernel , ; If the direction information of the current pixel is diagonal, the convolution kernel of the anisotropic Gaussian kernel , ; If the direction information of the current pixel is the anti-diagonal direction, the convolution kernel of the anisotropic Gaussian kernel , .in, Represents the unit vector of the current elliptical convolution kernel along the edge direction (long axis direction), Representation and A unit vector with a vertical direction (minor axis direction).

[0050] Afterwards, the intermediate matrix of the current pixel is calculated using the convolution kernel, edge strength and direction information. In this embodiment, the intermediate matrix O of the current pixel can be expressed as:

[0051] in, and Represent the vectors of the major axis and minor axis of the elliptical convolution kernel respectively; and Respectively represent and Vectors with perpendicular directions; matrices For the matrix The transpose of represents the edge strength, ; is an adjustable parameter; m is the direction credibility, It is a set of arrays, confirmed by direction credibility The value of .

[0052] The larger the value of , the faster the weight in the convolution kernel decreases with distance, the clearer the interpolation result is, and the greater the noise; different values ​​of directional credibility correspond to different Value, It is used to control the degree of interpolation extension according to the credibility of the edge direction. The smaller the value, the faster the weight of the direction deviating from the edge decreases, and the clearer the image edge is. However, the error is more obvious when the direction judgment is wrong. In practical applications, it is necessary to reasonably set it according to actual needs. and The value of .

[0053] Finally, the weight factor of the current pixel is calculated using the intermediate matrix and the distance vector between the current pixel and the surrounding pixels. , expressed as:

[0054] in, Represents the distance vector between the current pixel and the surrounding pixels, and O represents the intermediate matrix.

[0055] In practical applications, an M×M second cache window can be extracted, and the distance vector is calculated only for the pixels in the second cache window to obtain the weight factor of the current pixel with respect to each pixel in the second cache window, where M is a positive integer greater than or equal to 3 and can be selected according to actual needs.

[0056] In a specific embodiment, if Figure 5 As shown, a 5×5 second cache window is extracted; the current pixel is a B pixel, located at the 3rd row and 3rd column (located at the middle row and middle column of the second cache window). In the second cache window, in addition to B pixels, there are also R pixels and G pixels. Therefore, it is necessary to calculate the distance vector between each R pixel and each G pixel and the current B pixel. For example, the distance vector between the G pixel in the 2nd row and 3rd column and the current pixel can be expressed as [1,0]; the distance vector between the G pixel in the 3rd row and 1st column and the current pixel can be expressed as [0,-2]; the distance vector between the R pixel in the 1st row and 1st column and the current pixel can be expressed as [2,-2]; the distance vector between the R pixel in the 5th row and 5th column and the current pixel can be expressed as [2,2]. In this way, the weight factor of the current pixel with respect to each pixel in the second cache window can be obtained. ,like Figure 6 shown.

[0057] When the center pixel in the second cache window is an R pixel and a G pixel, the weight factor is calculated in the same manner and will not be described in detail in this application. When calculating the weight factor, the other three pixels in the four Bayer pixels where the current pixel is located do not need to calculate their weight factors because they are the same color pixels as the current pixel.

[0058] In the subsequent steps, if the pixel point corresponding to the weight factor is a G pixel point, it is used to interpolate and calculate the G channel value of the current pixel point; if the pixel point corresponding to the weight factor is an R pixel point or a B pixel point, it is used to interpolate and calculate the GR color difference value or GB color difference value of the current pixel point.

[0059] Furthermore, in this embodiment, in step S4, the method of calculating the G channel value, GR color difference value and GB color difference value of the current pixel using the weight factor includes: If the current pixel is an R pixel or a B pixel, the weight factor is used to perform weighted average calculation on the G pixels around it to obtain the G channel value of the current pixel.

[0060] Still Figure 5 As shown in the figure, for example, the current pixel is a B pixel, then the current pixel needs to use the weight factor to interpolate the surrounding G pixels to obtain the G channel value of the current pixel, which can be expressed as:

[0061] The calculation process of R pixels is similar, and this application will not repeat it. In this way, the G channel values ​​of all R pixels and B pixels in the four-Bayer format RAW image can be interpolated.

[0062] After the G channel values ​​of the R pixel and the B pixel are obtained, the color difference calculation can be performed using the pixel values ​​of the R pixel and the B pixel and the G channel value obtained by interpolation calculation to obtain the GR color difference value of the R pixel and the GB color difference value of the B pixel.

[0063] If the current pixel is a G pixel, the GR color difference value and GB color difference value of the current pixel need to be calculated by interpolation using the GR color difference value of the surrounding R pixels or the GB color difference value of the B pixels. Figure 7 As shown, the current pixel is a G pixel, located in the middle row and middle column of the 5×5 second cache window, and its adjacent pixel in the column direction is an R pixel, and its adjacent pixel in the row direction is a B pixel. The GR color difference value of the adjacent R pixel in the column direction and its corresponding weight factor are used to calculate the GR color difference value by weighted average; the GB color difference value of the adjacent B pixel in the row direction and its corresponding weight factor are used to calculate the GB color difference value by weighted average. The calculation method of the weight factor refers to the above step S3, and this application will not repeat it.

[0064] Similarly, if the current pixel is a B pixel, the weight factor is used to perform a weighted average calculation on the GR color difference values ​​of the R pixels around the current pixel to obtain the GR color difference value of the B pixel. Specifically, Figure 8As shown, the current pixel is a B pixel, located in the middle row and middle column of the 5×5 second cache window. In the second cache window, the four corners are R pixels. Therefore, the GR color difference value of the R pixel in the cache window and its corresponding weight factor are used to calculate the GR color difference value by weighted average. The calculation method of the weight factor refers to the above step S3, which will not be repeated in this application.

[0065] Similarly, when the current pixel is an R pixel, the GB color difference value of the current pixel can be calculated.

[0066] So far, the G channel values, GR color difference values, and GB color difference values ​​of all R pixels, the G channel values, GR color difference values, and GB color difference values ​​of all B pixels, and the GR color difference values ​​and GB color difference values ​​of all G pixels in the four-Bayer RAW image have been calculated.

[0067] Further, in this embodiment, in step S5, the method of obtaining the G channel value, the R channel value and the B channel value of the current pixel by using the G channel value, the GR color difference value and the GB color difference value includes: If the current pixel is a G pixel, the pixel value of the current pixel and the calculated GB color difference value and GR color difference value are subtracted to obtain the B channel value and the R channel value; If the current pixel is an R pixel, the G channel value calculated for the current pixel and the GB color difference value are subtracted to obtain the B channel value; If the current pixel is a B pixel, a difference calculation is performed on the G channel value and the GR color difference value calculated for the current pixel to obtain an R channel value.

[0068] So far, the G channel values ​​and B channel values ​​of all R pixels, the G channel values ​​and R channel values ​​of all B pixels, and the R channel values ​​and B channel values ​​of all G pixels in the four-Bayer RAW image are calculated.

[0069] Further, in this embodiment, S6, the G channel value, the R channel value and the B channel value of the current pixel are combined to output an RGB image.

[0070] Specifically, for R pixels, their own pixel values ​​(R channel values) are combined with the G channel values ​​and B channel values ​​obtained by interpolation; for B pixels, their own pixel values ​​(B channel values) are combined with the G channel values ​​and R channel values ​​obtained by interpolation; for G pixels, their own pixel values ​​(G channel values) are combined with the R channel values ​​and B channel values ​​obtained by interpolation. In this way, the final RGB image can be output.

[0071] This embodiment also provides a four-Bayer format RAW image interpolation device, which is used to implement the four-Bayer format RAW image interpolation method as described above. Fig. 9 As shown, the interpolation device of the four-Bayer format RAW image includes: An image acquisition module, used for acquiring a four-Bayer format RAW image; Gradient calculation module, used to calculate the gradient information of the current pixel; The weight factor calculation module is used to calculate the weight factor of the current pixel point according to the gradient information; The color difference calculation module is used to calculate the G channel value, GR color difference value and GB color difference value of the current pixel point using the weight factor; The full channel calculation module is used to obtain the G channel value, R channel value and B channel value of the current pixel point by using the G channel value, GR color difference value and GB color difference value; The image output module is used to combine the G channel value, R channel value and B channel value of the current pixel point to output an RGB image.

[0072] The interpolation device for a four-Bayer format RAW image provided in this embodiment calculates the gradient information of the current pixel point through an image interpolation module, and then calculates the weight factor, so as to obtain the edge information of the image; the RGB channel value of the current pixel point is calculated through the weight factor, so as to obtain an image conversion result with low noise in a flat area and clear details in an edge area. Not only does it not require the existing demosaicing algorithm, but it also realizes the conversion of the four-Bayer format RAW image to the RGB image, and also ensures the image quality, solving the problem of how to convert the four-Bayer format RAW image into a high-quality color image.

[0073] Furthermore, this embodiment further provides an electronic device, comprising a memory, a processor, and an executable program stored in the memory and capable of being run by the processor; when the processor runs the executable program, the interpolation method of the four-Bayer format RAW image as described above is executed.

[0074] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other. In addition, the different parts between the various embodiments can also be used in combination with each other, and the present invention is not limited to this.

[0075] The interpolation method and device of the four-Bayer format RAW image provided in this embodiment, and the electronic device, include: obtaining the four-Bayer format RAW image; calculating the gradient information of the current pixel; calculating the weight factor of the current pixel according to the gradient information; using the weight factor, calculating the G channel value, GR color difference value and GB color difference value of the current pixel; using the G channel value, GR color difference value and GB color difference value, obtaining the G channel value, R channel value and B channel value of the current pixel; combining the G channel value, R channel value and B channel value of the current pixel to output an RGB image. By calculating the gradient information of the current pixel and then calculating the weight factor, the edge information of the image can be obtained; by calculating the RGB channel value of the current pixel through the weight factor, the image conversion result with low noise in the flat area and clear details in the edge area can be obtained, which not only does not require the existing de-mosaicing algorithm, but also realizes the conversion of the four-Bayer format RAW image to the RGB image, and also ensures the image quality, solving the problem of how to convert the four-Bayer format RAW image into a high-quality color image.

[0076] The above description is only a description of the preferred embodiments of the present invention, and is not intended to limit the scope of the present invention. Any changes or modifications made by a person skilled in the art in the field of the present invention based on the above disclosure shall fall within the scope of protection of the claims.

Claims

1. A method for interpolating a RAW image in a four-Bayer format, characterized in that: include: Get quad-Bayer format RAW images; Calculate the gradient information of the current pixel; According to the gradient information, calculate the weight factor of the current pixel; Using the weight factor, calculate the G channel value, GR color difference value and GB color difference value of the current pixel; Using the G channel value, GR color difference value and GB color difference value, the G channel value, R channel value and B channel value of the current pixel are obtained; Combine the G channel value, R channel value, and B channel value of the current pixel to output an RGB image.

2. The interpolation method of a quad-Bayer format RAW image according to claim 1, characterized in that: The method for calculating the gradient information of the current pixel point includes: The horizontal gradient of the current pixel is calculated using the four-Bayer pixel where the current pixel is located. , vertical gradient , diagonal gradient and the anti-diagonal gradient , expressed as: in, Represents the pixel value of the pixel located in the upper left corner of the four-Bayer pixel, Represents the pixel value of the pixel located in the upper right corner of the four-Bayer pixel, Represents the pixel value of the pixel at the lower left corner of the four-Bayer pixel, Indicates the pixel value of the pixel located at the lower right corner in the four-Bayer pixel; the gradient information of the four pixels in the same four-Bayer pixel is consistent.

3. The interpolation method of a quad-Bayer format RAW image according to claim 2, characterized in that: The method for calculating the weight factor of the current pixel point according to the gradient information includes: According to the horizontal gradient of the current pixel , vertical gradient , diagonal gradient and the anti-diagonal gradient The minimum value among them confirms the direction information of the current pixel; According to the horizontal gradient of the current pixel , vertical gradient , diagonal gradient and the anti-diagonal gradient The maximum value among them confirms the edge strength of the current pixel; Calculate the weight factor of the current pixel based on the direction information and edge strength of the current pixel.

4. The interpolation method of a quad-Bayer format RAW image according to claim 3, characterized in that: The method for calculating the weight factor of the current pixel point according to the direction information and edge strength of the current pixel point includes: Count the direction information of all four Bayer pixels to obtain a direction matrix; The number of pixels in the direction matrix that are consistent with the direction information of the current pixel is counted to obtain the direction credibility of the current pixel. According to the direction information of the current pixel, the convolution kernel of the current pixel is determined; Using the convolution kernel, edge strength and direction information, calculate the intermediate matrix of the current pixel; The weight factor of the current pixel is calculated using the intermediate matrix and the distance vector between the current pixel and the surrounding pixels.

5. The interpolation method of a quad-Bayer format RAW image according to claim 4, characterized in that: The method for determining the convolution kernel of the current pixel point according to the direction information of the current pixel point includes: Construct an anisotropic Gaussian kernel where: If the horizontal gradient of the current pixel , vertical gradient , diagonal gradient and the anti-diagonal gradient The minimum value in is the horizontal gradient, so the direction information of the current pixel is the horizontal direction, and the convolution kernel of the anisotropic Gaussian kernel , ; If the horizontal gradient of the current pixel , vertical gradient , diagonal gradient and the anti-diagonal gradient The minimum value in is the vertical gradient, so the direction information of the current pixel is the vertical direction, and the convolution kernel of the anisotropic Gaussian kernel is , ; If the horizontal gradient of the current pixel , vertical gradient , diagonal gradient and the anti-diagonal gradient The minimum value in is the diagonal gradient, so the direction information of the current pixel is the diagonal direction, and the convolution kernel of the anisotropic Gaussian kernel , ; If the horizontal gradient of the current pixel , vertical gradient , diagonal gradient and the anti-diagonal gradient The minimum value in is the anti-diagonal gradient, so the direction information of the current pixel is the anti-diagonal direction, and the convolution kernel of the anisotropic Gaussian kernel , .

6. The interpolation method of a quad-Bayer format RAW image according to claim 4, characterized in that: The convolution kernel, edge strength and direction information are used to calculate the intermediate matrix O of the current pixel, which is expressed as: in, and Represent the vectors of the major axis and minor axis of the elliptical convolution kernel respectively; and Respectively represent and A vector with a perpendicular direction; represents the edge strength, ; is an adjustable parameter; m is the direction credibility, It is a set of arrays, confirmed by direction credibility The value of .

7. The interpolation method of a quad-Bayer format RAW image according to claim 4, characterized in that: The weight factor of the current pixel is calculated by using the intermediate matrix and the distance vector between the current pixel and the surrounding pixels. , expressed as: in, Represents the distance vector between the current pixel and the surrounding pixels, and O represents the intermediate matrix.

8. The interpolation method of a quad-Bayer format RAW image according to claim 1, characterized in that: The method of calculating the G channel value, GR color difference value and GB color difference value of the current pixel point by using the weight factor includes: If the current pixel is an R pixel or a B pixel, the weight factor is used to perform weighted average calculation on the G pixels around it to obtain the G channel value of the current pixel; If the current pixel is an R pixel or a B pixel, the color difference is calculated using the pixel value of the current pixel and the G channel value obtained by interpolation to obtain the GR color difference value of the R pixel and the GB color difference value of the B pixel; If the current pixel is an R pixel, the weight factor is used to perform weighted average calculation on the GB color difference values ​​of the B pixels around the current pixel to obtain the GB color difference value of the R pixel; if the current pixel is a B pixel, the weight factor is used to perform weighted average calculation on the GR color difference values ​​of the R pixels around the current pixel to obtain the GR color difference value of the B pixel; if the current pixel is a G pixel, the weight factor is used to perform weighted average calculation on the GB color difference values ​​of the B pixels and the GR color difference values ​​of the R pixels around the current pixel to obtain the GB color difference value and GR color difference value of the G pixel.

9. The method for interpolating a RAW image in a quad-Bayer format according to claim 1, wherein: The method of obtaining the G channel value, the R channel value and the B channel value of the current pixel by using the G channel value, the GR color difference value and the GB color difference value comprises: If the current pixel is a G pixel, the pixel value of the current pixel and the calculated GB color difference value and GR color difference value are subtracted to obtain the B channel value and the R channel value; If the current pixel is an R pixel, the G channel value calculated for the current pixel and the GB color difference value are subtracted to obtain the B channel value; If the current pixel is a B pixel, a difference calculation is performed on the G channel value and the GR color difference value calculated for the current pixel to obtain an R channel value.

10. A device for interpolating a RAW image in a four-Bayer format, used to implement the interpolation method for a RAW image in a four-Bayer format as claimed in any one of claims 1 to 9, characterized in that: The quad-Bayer format RAW image interpolation device comprises: An image acquisition module, used for acquiring a four-Bayer format RAW image; Gradient calculation module, used to calculate the gradient information of the current pixel; The weight factor calculation module is used to calculate the weight factor of the current pixel point according to the gradient information; The color difference calculation module is used to calculate the G channel value, GR color difference value and GB color difference value of the current pixel point using the weight factor; The full channel calculation module is used to obtain the G channel value, R channel value and B channel value of the current pixel point by using the G channel value, GR color difference value and GB color difference value; The image output module is used to combine the G channel value, R channel value and B channel value of the current pixel point to output an RGB image.

11. An electronic device, characterized in that: The invention comprises a memory, a processor and an executable program stored in the memory and capable of being run by the processor; when the processor runs the executable program, the interpolation method of the four-Bayer format RAW image as claimed in any one of claims 1 to 9 is executed.

Images