Implementation Method and Device for Re-Mosaic

Abstract

The present invention discloses a method and apparatus for realizing remosaicking, including: calculating gradients of a unit where a pixel point that needs green interpolation in a multi-unit CFA image is located in multiple directions and calculating a texture gradient matrix of the unit and adjacent units; calculating a texture similarity coefficient according to the texture gradient matrix of the unit and adjacent units, determining an interpolation direction according to the texture similarity coefficient and gradients in each direction, and performing green component interpolation on the pixel points of the unit; performing blue component interpolation and red component interpolation on the pixel points at the positions of red and blue components respectively according to the interpolated full-resolution green channel, and performing red and blue component interpolation on the pixel points at the position of the green component according to the interpolated blue and red channels; combining all color channels to obtain a remosaicked image. It can be seen that the present invention can reduce the algorithm complexity and hardware overhead, is beneficial to hardware implementation, and the processing flow has real-time performance.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a method and device for realizing remosaicking. Background Art

[0002] In order to reduce costs, decrease volume, and facilitate hardware processing, the raw image captured by a single image sensor (Sensor) is called a Color Filter Array (CFA) image. The CFA image is a mosaic-arranged image. The color filter array covered in front of the single image sensor filters the captured light according to wavelength. This method can regularly collect only one color component among multiple color components at each pixel point of the image, and there is a corresponding color component arrangement for a specific CFA type. Then, the Image Signal Processor (ISP) processes the CFA image captured by the single image sensor. After the demosaicking operation, a complete full-color image is obtained. Among them, the process of reconstructing the missing color components at each pixel point in the CFA image so that their values can be associated with each pixel point is called demosaicking (or interpolation).

[0003] With the evolution of technology, the resolution of image sensors is getting higher and higher. In order to increase the signal-to-noise ratio, multi-cell color filter arrays (multi-cell CFA) have emerged. Each cell of the multi-cell CFA (including N*N pixel points) has the same color components. For example, Figure 1 the four-cell CFA in which every 2*2 pixel points form a cell as shown in Fig. 1a, Figure 1 the nine-cell CFA in which every 3*3 pixel points form a cell as shown in Fig. 1b, and Figure 1 the sixteen-cell CFA in which every 4*4 pixel points form a cell as shown in Fig. 1c. It has been found in practice that in order to enable the demosaicking module of the image signal processor of the traditional one-cell color filter array (one-cell CFA) to compatibly process the images captured by the multi-cell CFA (i.e., multi-cell CFA images), in the existing implementation methods, interpolation of the green channel is performed in eight directions. This method has problems such as high algorithm complexity and large hardware overhead. Or, in the existing implementation methods, the multi-cell CFA image is split into multiple low-resolution Bayer-format CFA images with the same arrangement pattern, the demosaicking process is performed on these low-resolution Bayer-format CFA images to become RGB images, and finally they are fused into a full-resolution RGB image. This method also has problems such as high algorithm complexity and large hardware overhead. It can be seen that the current methods for the image signal processor to process multi-cell CFA images have problems such as high algorithm complexity and large hardware overhead, which are not conducive to hardware implementation. Summary of the Invention

[0004] The present invention provides a method and device for realizing remosaicking, which can reduce the algorithm complexity and hardware overhead while processing a multi-unit CFA image to ensure the visual effect of the image, and is beneficial to hardware implementation.

[0005] The first aspect of the present invention discloses a method for realizing remosaicking, and the method includes:

[0006] Obtain a multi-unit CFA image to be processed, calculate the gradients of the unit where the current pixel point that needs to perform green channel interpolation in the multi-unit CFA image in multiple target directions, and calculate the texture gradient matrix of the unit where the current pixel point is located and the texture gradient matrix of each adjacent unit among all adjacent units of the unit where the current pixel point is located;

[0007] Calculate the texture similarity coefficient according to the texture gradient matrix of the unit where the current pixel point is located and the texture gradient matrix of each adjacent unit, determine the target interpolation direction of the unit where the current pixel point is located according to the texture similarity coefficient and the gradients of the unit where the current pixel point is located in each target direction, and perform green component interpolation operations on all pixel points of the unit where the current pixel point is located in the target interpolation direction;

[0008] Perform blue component interpolation operations on the pixel points at the determined red component positions and perform red component interpolation operations on the pixel points at the determined blue component positions according to the interpolated full-resolution green channel, and, perform red component interpolation operations and blue component interpolation operations on the pixel points at the determined green component positions according to the interpolated blue channel and red channel;

[0009] Combine all the color channels obtained by interpolation to obtain a remosaicked image.

[0010] As an optional implementation manner, in the first aspect of the present invention, when the determined red component positions are partial red component positions, the determined blue component positions are partial blue component positions, and the determined green component positions are partial green component positions, perform a demosaicking operation on the remosaicked image to obtain a full-color image;

[0011] When the determined red component positions are all red component positions, the determined blue component positions are all blue component positions, and the determined green component positions are all green component positions, perform a pseudo-color removal operation on the remosaicked image to obtain a full-color image.

[0012] As an optional implementation manner, in the first aspect of the present invention, the method further includes:

[0013] After obtaining the full-color image, convert the full-color image into an image in a target format, and output the image in the target format;

[0014] Among them, all the target directions include the horizontal direction, the vertical direction, the diagonal direction, and the anti-diagonal direction.

[0015] As an optional implementation manner, in the first aspect of the present invention, determining the target interpolation direction of the unit where the current pixel point is located according to the texture similarity coefficient and the gradient of the unit where the current pixel point is located in each of the target directions includes:

[0016] According to the gradient of the unit where the current pixel point is located in each of the target directions, the texture similarity coefficient, and the weight coefficient corresponding to each of the target directions, calculate the value corresponding to each of the target directions;

[0017] According to the target minimum value among the values corresponding to all the target directions, a pre-determined threshold value, and the remaining values among the values corresponding to all the target directions except the target minimum value, determine the target interpolation direction of the unit where the current pixel point is located.

[0018] As an optional implementation manner, in the first aspect of the present invention, determining the target interpolation direction of the unit where the current pixel point is located according to the target minimum value among the values corresponding to all the target directions, a pre-determined threshold value, and the remaining values among the values corresponding to all the target directions except the target minimum value includes:

[0019] Judge whether the sum of the target minimum value among the values corresponding to all the target directions and the pre-determined threshold value is less than the minimum value among all the remaining values, and all the remaining values include the values among the values corresponding to all the target directions except the target minimum value;

[0020] When the judgment result is yes, determine the target direction corresponding to the target minimum value as the target interpolation direction of the unit where the current pixel point is located;

[0021] When the judgment result is no, determine the empty direction as the target interpolation direction of the unit where the current pixel point is located.

[0022] As an optional implementation manner, in the first aspect of the present invention, performing a green component interpolation operation on all the pixel points of the unit where the current pixel point is located in the target interpolation direction includes:

[0023] According to the target interpolation direction, determine an interpolation green component calculation formula that matches the target interpolation direction, and obtain calculation parameters corresponding to the interpolation green component calculation formula;

[0024] According to the interpolation green component calculation formula and the calculation parameters corresponding to the interpolation green component calculation formula, calculate the interpolation green component corresponding to each pixel in the unit where the current pixel is located, and perform a green component interpolation operation on the pixel in the target interpolation direction according to the interpolation green component corresponding to each pixel in the unit where the current pixel is located.

[0025] As an optional implementation manner, in the first aspect of the present invention, when the determined red component position is a partial red component position, the determined blue component position is a partial blue component position, and the determined green component position is a partial green component position, before performing a blue component interpolation operation on the pixel at the determined red component position and performing a red component interpolation operation on the pixel at the determined blue component position according to the interpolated full-resolution green channel, the method further includes:

[0026] According to the determined remosaicking pattern, determine the specified red component position in the unit of the red component as the red component position where a blue component interpolation operation needs to be performed, and determine the specified blue component position in the unit of the blue component as the blue component position where a red component interpolation operation needs to be performed;

[0027] And, before performing a red component interpolation operation and a blue component interpolation operation on the pixel at the determined green component position according to the interpolated blue channel and red channel, the method further includes:

[0028] According to the remosaicking pattern, determine the specified first green component position in the unit of the green component as the green component position where a blue component interpolation operation needs to be performed and determine the specified second green component position in the unit of the green component as the green component position where a red component interpolation operation needs to be performed;

[0029] Wherein, combining all the color channels obtained by interpolation to obtain a remosaicked image includes:

[0030] Combine all the color channels obtained by interpolation according to the remosaicking pattern to obtain a mosaic image in a target arrangement manner.

[0031] The second aspect of the present invention discloses an apparatus for realizing remosaicking, and the apparatus includes:

[0032] An image acquisition module, configured to acquire a multi-unit CFA image to be processed;

[0033] A remosaicking module is configured to calculate gradients of the unit where the current pixel point that needs to perform green channel interpolation in the multi-unit CFA image is located in multiple target directions, calculate the texture gradient matrix of the unit where the current pixel point is located, and the texture gradient matrix of each of all adjacent units of the unit where the current pixel point is located; and, calculate a texture similarity coefficient according to the texture gradient matrix of the unit where the current pixel point is located and the texture gradient matrix of each adjacent unit, determine the target interpolation direction of the unit where the current pixel point is located according to the texture similarity coefficient and the gradients of the unit where the current pixel point is located in each of the target directions, and perform a green component interpolation operation on all pixel points of the unit where the current pixel point is located in the target interpolation direction; and, perform a blue component interpolation operation on the pixel points at the determined red component positions and perform a red component interpolation operation on the pixel points at the determined blue component positions according to the interpolated full-resolution green channel, and, perform a red component interpolation operation and a blue component interpolation operation on the pixel points at the determined green component positions according to the interpolated blue channel and red channel;

[0034] A combination module is configured to combine all the color channels obtained by interpolation to obtain a remosaicked image.

[0035] As an optional implementation manner, in the second aspect of the present invention, the apparatus further includes:

[0036] A demosaicking module is configured to perform a demosaicking process on the remosaicked image to obtain a full-color image when the determined red component positions are partial red component positions, the determined blue component positions are partial blue component positions, and the determined green component positions are partial green component positions.

[0037] As an optional implementation manner, in the second aspect of the present invention, the remosaicking module is further configured to perform a pseudo-color removal operation on the remosaicked image to obtain a full-color image when the determined red component positions are all red component positions, the determined blue component positions are all blue component positions, and the determined green component positions are all green component positions.

[0038] As an optional implementation manner, in the second aspect of the present invention, the apparatus further includes:

[0039] An image output module is configured to convert the full-color image into an image in a target format and output the image in the target format after obtaining the full-color image;

[0040] Wherein, all the target directions include a horizontal direction, a vertical direction, a diagonal direction, and an anti-diagonal direction.

[0041] As an alternative implementation manner, in the second aspect of the present invention, the specific manner in which the remosaicking module determines the target interpolation direction of the unit where the current pixel is located according to the texture similarity coefficient and the gradient of the unit where the current pixel is located in each of the target directions is as follows:

[0042] Calculate the value corresponding to each of the target directions according to the gradient of the unit where the current pixel is located in each of the target directions, the texture similarity coefficient, and the weight coefficient corresponding to each of the target directions;

[0043] Determine the target interpolation direction of the unit where the current pixel is located according to the target minimum value among all the values corresponding to the target directions, a pre-determined threshold value, and the remaining values among all the values corresponding to the target directions except the target minimum value.

[0044] As an alternative implementation manner, in the second aspect of the present invention, the specific manner in which the remosaicking module determines the target interpolation direction of the unit where the current pixel is located according to the target minimum value among all the values corresponding to the target directions, a pre-determined threshold value, and the remaining values among all the values corresponding to the target directions except the target minimum value is as follows:

[0045] Judge whether the sum of the target minimum value among all the values corresponding to the target directions and the pre-determined threshold value is less than the minimum value among all the remaining values, and all the remaining values include the values among all the values corresponding to the target directions except the target minimum value;

[0046] When the judgment result is yes, determine the target direction corresponding to the target minimum value as the target interpolation direction of the unit where the current pixel is located;

[0047] When the judgment result is no, determine the empty direction as the target interpolation direction of the unit where the current pixel is located.

[0048] As an alternative implementation manner, in the second aspect of the present invention, the specific manner in which the remosaicking module performs a green component interpolation operation on all the pixels of the unit where the current pixel is located in the target interpolation direction is as follows:

[0049] According to the target interpolation direction, determine an interpolation green component calculation formula that matches the target interpolation direction, and obtain calculation parameters corresponding to the interpolation green component calculation formula;

[0050] According to the interpolation green component calculation formula and the calculation parameters corresponding to the interpolation green component calculation formula, calculate the interpolation green component corresponding to each pixel in the unit where the current pixel is located, and perform a green component interpolation operation on the pixel in the target interpolation direction according to the interpolation green component corresponding to each pixel in the unit where the current pixel is located.

[0051] As an optional implementation manner, in the second aspect of the present invention, the remosaicking module is further configured to, when the determined red component position is a partial red component position, the determined blue component position is a partial blue component position, and the determined green component position is a partial green component position, and before performing a blue component interpolation operation on the pixel at the determined red component position and a red component interpolation operation on the pixel at the determined blue component position according to the interpolated full-resolution green channel, determine, according to the determined remosaicking mode, the specified red component position in the unit of the red component as the red component position where the blue component interpolation operation needs to be performed, and determine the specified blue component position in the unit of the blue component as the blue component position where the red component interpolation operation needs to be performed;

[0052] The remosaicking module is further configured to, when the determined red component position is a partial red component position, the determined blue component position is a partial blue component position, and the determined green component position is a partial green component position, and before performing a red component interpolation operation and a blue component interpolation operation on the pixel at the determined green component position according to the interpolated blue channel and red channel, determine, according to the remosaicking mode, the specified first green component position in the unit of the green component as the green component position where the blue component interpolation operation needs to be performed and the specified second green component position in the unit of the green component as the green component position where the red component interpolation operation needs to be performed;

[0053] Wherein, the specific manner in which the combining module combines all the interpolated color channels to obtain a remosaicked image is as follows:

[0054] Combine all the interpolated color channels according to the remosaicking mode to obtain a mosaic image in a target arrangement manner.

[0055] A third aspect of the present invention discloses another device for implementing remosaicking, the device includes:

[0056] A memory storing executable program code;

[0057] A processor coupled to the memory;

[0058] The processor calls the executable program code stored in the memory and executes some or all of the steps in the method for implementing remosaicking disclosed in the first aspect of the present invention.

[0059] The fourth aspect of the present invention discloses a computer storage medium. The computer storage medium stores computer instructions, which are used to execute some or all of the steps in the method for implementing remosaicking disclosed in the first aspect of the present invention when called.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] The present invention can calculate the gradients of the unit where the pixel points that need to perform green interpolation in the multi-unit CFA image are located in multiple directions and calculate the texture gradient matrix of the unit and its adjacent units; calculate the texture similarity coefficient according to the texture gradient matrix of the unit and its adjacent units, determine the interpolation direction according to the texture similarity coefficient and the gradients in each direction, and perform green component interpolation on the pixel points of the unit; perform blue component interpolation and red component interpolation on the pixel points at the positions of the red and blue components respectively according to the interpolated full-resolution green channel, and perform red and blue component interpolation operations on the pixel points at the positions of the green component according to the interpolated blue and red channels; combine all color channels to obtain a remosaicked image. It can be seen that the present invention can enable the image signal processor to compatibly process multi-unit CFA images, reduce the algorithm complexity and hardware overhead while restoring the missing parts in the multi-unit CFA image to ensure the visual effect of the image, facilitate hardware implementation, and the processing flow has real-time performance. Description of the Drawings

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0063] Figure 1 It is a schematic structural diagram of a multi-unit CFA image disclosed in an embodiment of the present invention;

[0064] Figure 2 It is a schematic flowchart of a method for implementing remosaicking disclosed in an embodiment of the present invention;

[0065] Figure 3 It is a schematic flowchart of another method for implementing remosaicking disclosed in an embodiment of the present invention;

[0066] Figure 4 It is a schematic diagram of the result of a mosaic image transformation disclosed in an embodiment of the present invention;

[0067] Figure 5 It is a schematic diagram of the result of another mosaic image transformation disclosed in the embodiments of the present invention;

[0068] Figure 6 It is a schematic diagram of a CFA image where a blue pixel and its corresponding unit are located at the center of the window disclosed in the embodiments of the present invention;

[0069] Figure 7 It is a schematic diagram of a CFA image where a red pixel and its corresponding unit are located at the center of the window disclosed in the embodiments of the present invention;

[0070] Figure 8 It is a schematic diagram of the result of interpolating the corresponding color components at the center position disclosed in the embodiments of the present invention;

[0071] Figure 9 It is a schematic diagram of a CFA image where a green pixel and its corresponding unit are located at the center of the window disclosed in the embodiments of the present invention;

[0072] Figure 10 It is a schematic diagram of the result of interpolating the red and blue components in the unit where the green pixel is located disclosed in the embodiments of the present invention;

[0073] Figure 11 It is a schematic diagram of the result of synthesizing a mosaic image by combining the red, green, and blue channels disclosed in the embodiments of the present invention;

[0074] Figure 12 It is a schematic diagram of a CFA image in the Quad RGBW format and the Quad RGBIR format disclosed in the embodiments of the present invention;

[0075] Figure 13 It is a schematic diagram of a CFA image in the Quad RGBW format and the Quad RGBIR format with a minimum sampling period of 4*4 disclosed in the embodiments of the present invention;

[0076] Figure 14 It is a schematic diagram of the structure of an implementation device for remosaicking disclosed in the embodiments of the present invention;

[0077] Figure 15 It is a schematic diagram of the structure of another implementation device for remosaicking disclosed in the embodiments of the present invention;

[0078] Figure 16 It is a schematic diagram of the structure of yet another implementation device for remosaicking disclosed in the embodiments of the present invention. Detailed implementation manners

[0079] To enable those skilled in the art to better understand the solution of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0080] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product, port or terminal comprising a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products, ports or terminals.

[0081] Referring to "embodiment" herein means that a specific feature, structure or characteristic described in connection with the embodiment may be included in at least one embodiment of the present invention. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein may be combined with other embodiments.

[0082] The present invention discloses a method and device for realizing remosaicking, which can enable an image signal processor to compatibly process multi-unit CFA images. While restoring the missing parts in the multi-unit CFA images to ensure the visual effect of the images, it reduces the algorithm complexity and hardware overhead, is conducive to hardware implementation, and the processing flow has real-time performance. In addition, it also supports CFA images in various formats, improving the flexibility of remosaicking processing for multi-unit CFA images. The following will be described in detail respectively.

[0083] Embodiment 1

[0084] Please refer to Figure 2 , Figure 2 which is a schematic flowchart of a method for realizing remosaicking disclosed in an embodiment of the present invention. Among them, Figure 2 the described method can be applied to an image signal processor. As Figure 2 shown, the method may include the following operations:

[0085] 101. The image signal processor acquires a multi-unit CFA image to be processed.

[0086] In the embodiments of the present invention, the multi-unit CFA image is any multi-unit CFA image to be processed.

[0087] 102. The image signal processor calculates the gradients of the unit where the current pixel point that needs to perform green channel interpolation in the multi-unit CFA image in multiple target directions.

[0088] In the embodiments of the present invention, the multiple target directions can be one of 2 directions, 4 directions, 6 directions, 8 directions, etc., and the embodiments of the present invention do not make limitations.

[0089] 103. The image signal processor calculates the texture gradient matrix of the unit where the current pixel point is located and the texture gradient matrix of each adjacent unit among all adjacent units of the unit where the current pixel point is located.

[0090] 104. The image signal processor calculates the texture similarity coefficient according to the texture gradient matrix of the unit where the current pixel point is located and the texture gradient matrix of each adjacent unit, and determines the target interpolation direction of the unit where the current pixel point is located according to the texture similarity coefficient and the gradient of the unit where the current pixel point is located in each target direction.

[0091] 105. The image signal processor performs green component interpolation operations on all pixel points of the unit where the current pixel point is located in the target interpolation direction.

[0092] 106. The image signal processor performs blue component interpolation operations on the pixel points at the determined red component positions and performs red component interpolation operations on the pixel points at the determined blue component positions according to the interpolated full-resolution green channel.

[0093] 107. The image signal processor performs red component interpolation operations and blue component interpolation operations on the pixel points at the determined green component positions according to the interpolated blue channel and red channel.

[0094] 108. The image signal processor combines all the interpolated color channels to obtain a demosaicked image. It can be seen that implementing the method described in the embodiments of the present invention can enable the image signal processor to compatibly process multi-unit CFA images, while restoring the missing parts in the multi-unit CFA images to ensure the visual effect of the images, reducing the algorithm complexity and hardware overhead, being conducive to hardware implementation, and the processing flow has real-time performance.

[0095] Embodiment 2

[0096] Please refer to Figure 3 , Figure 3 which is a schematic flowchart of another implementation method of demosaicking disclosed in the embodiments of the present invention. Among them, Figure 3 the described method can be applied to an image signal processor. AsFigure 3 As shown, the method may include the following operations:

[0097] 201. An image signal processor acquires a multi-unit CFA image to be processed.

[0098] 202. The image signal processor calculates the gradients of the unit where the current pixel point that needs to perform green channel interpolation in the multi-unit CFA image in multiple target directions.

[0099] Optionally, the multiple target directions may be 4 directions, specifically including the horizontal direction, the vertical direction, the diagonal direction, and the anti-diagonal direction.

[0100] 203. The image signal processor calculates the texture gradient matrix of the unit where the current pixel point is located and the texture gradient matrix of each adjacent unit among all adjacent units of the unit where the current pixel point is located.

[0101] It should be noted that there is no order of execution between step 202 and step 203.

[0102] 204. The image signal processor calculates the texture similarity coefficient according to the texture gradient matrix of the unit where the current pixel point is located and the texture gradient matrix of each adjacent unit, and determines the target interpolation direction of the unit where the current pixel point is located according to the texture similarity coefficient and the gradient of the unit where the current pixel point is located in each target direction.

[0103] 205. The image signal processor performs a green component interpolation operation on all pixel points of the unit where the current pixel point is located in the target interpolation direction.

[0104] 206. The image signal processor performs a blue component interpolation operation on the pixel points at the determined red component positions and a red component interpolation operation on the pixel points at the determined blue component positions according to the interpolated full-resolution green channel.

[0105] 207. The image signal processor performs a red component interpolation operation and a blue component interpolation operation on the pixel points at the determined green component positions according to the interpolated blue channel and red channel.

[0106] 208. The image signal processor combines all the interpolated color channels to obtain a remosaicked image.

[0107] Among them, for the detailed description of steps 201 - 208, please refer to the detailed description of steps 101 - 108 in Embodiment 1, and the embodiments of the present invention will not be elaborated herein.

[0108] In the embodiments of the present invention, as Figure 3 shown, the method further includes steps 209 - 211, where:

[0109] 209. The image signal processor performs a demosaicing operation on the remosaicked image to obtain a full-color image.

[0110] 210. The image signal processor performs a pseudo-color removal operation on the remosaicked image to obtain a full-color image.

[0111] 211. The image signal processor converts the full-color image into an image in a target format and outputs the image in the target format.

[0112] In the embodiments of the present invention, the image in the target format may be an image in RGB format or an image in YUV format, which is not limited in the embodiments of the present invention.

[0113] Among them, after step 208 is completed, when the determined red component position is a partial red component position, the determined blue component position is a partial blue component position, and the determined green component position is a partial green component position, step 209 is triggered to be executed; when the determined red component position is all red component positions, the determined blue component position is all blue component positions, and the determined green component position is all green component positions, step 210 is triggered to be executed.

[0114] For example, the mosaic image transformation result corresponding to the interpolation of the corresponding color component at the specified component position can be referred to Figure 4 as shown; the mosaic image transformation result corresponding to the interpolation of the corresponding color component at all component positions (i.e., the interpolation of the green channel at full resolution, the interpolation of the red and blue channels at full resolution) can be referred to Figure 5 as shown.

[0115] It can be seen that implementing the method described in the embodiments of the present invention can enable the image signal processor to compatibly process multi-unit CFA images. While restoring the missing parts in the multi-unit CFA images to ensure the visual effect of the images, it reduces the algorithm complexity and hardware overhead, is conducive to hardware implementation, and the processing flow has real-time performance. In addition, it also supports multi-unit CFA images in diverse formats, improving the flexibility of remosaicking multi-unit CFA images. In addition, it can not only restore the multi-unit CFA image to a complete full-resolution color image, but also transform the multi-unit CFA image into a mosaic image in the form of a single-unit CFA. In addition, it can also combine the processes of remosaicking and demosaicing, and only need to interpolate the corresponding color components at the corresponding positions. Moreover, it can also perform pseudo-color detection and removal operations during the demosaicing stage, which is conducive to saving computational overhead.

[0116] In an alternative embodiment, the image signal processor calculates a texture similarity coefficient based on the texture gradient matrix of the unit where the current pixel is located and the texture gradient matrices of each adjacent unit, and determines the target interpolation direction of the unit where the current pixel is located according to the texture similarity coefficient and the gradient of the unit where the current pixel is located in each target direction, which may include:

[0117] The image signal processor calculates a value corresponding to each target direction according to the gradient of the unit where the current pixel is located in each target direction, the texture similarity coefficient, and the weight coefficient corresponding to each target direction;

[0118] The image signal processor determines the target interpolation direction of the unit where the current pixel is located according to the target minimum value among the values corresponding to all target directions, a predetermined threshold, and the remaining values except the target minimum value among the values corresponding to all target directions.

[0119] Among them, the predetermined threshold can be adaptively adjusted according to the actual situation.

[0120] Further alternatively, the image signal processor determines the target interpolation direction of the unit where the current pixel is located according to the target minimum value among the values corresponding to all target directions, a predetermined threshold, and the remaining values except the target minimum value among the values corresponding to all target directions, which may include:

[0121] The image signal processor determines whether the sum of the target minimum value among the values corresponding to all target directions and the predetermined threshold is less than the minimum value among all the remaining values, and all the remaining values include the values except the target minimum value among the values corresponding to all target directions;

[0122] When the judgment result is yes, the image signal processor determines the target direction corresponding to the target minimum value as the target interpolation direction of the unit where the current pixel is located;

[0123] When the judgment result is no, the image signal processor determines the empty direction as the target interpolation direction of the unit where the current pixel is located.

[0124] It can be seen that this alternative embodiment can intelligently determine the interpolation direction of the unit where the current pixel is located according to the calculated value corresponding to each direction and the predetermined threshold, so as to realize green channel interpolation.

[0125] Still further alternatively, the image signal processor performs a green component interpolation operation on all pixels of the unit where the current pixel is located in the target interpolation direction, which may include:

[0126] The image signal processor determines an interpolation green component calculation formula that matches the target interpolation direction according to the target interpolation direction, and obtains calculation parameters corresponding to the interpolation green component calculation formula;

[0127] The image signal processor calculates the interpolated green component corresponding to each pixel in the unit where the current pixel is located according to the interpolated green component calculation formula and the calculation parameters corresponding to the interpolated green component calculation formula, and performs a green component interpolation operation on the pixel in the target interpolation direction according to the interpolated green component corresponding to each pixel in the unit where the current pixel is located.

[0128] It can be seen that this optional embodiment can also intelligently match the corresponding interpolated green component calculation formula and calculation parameters according to the determined interpolation direction, and then calculate the interpolated green component corresponding to each pixel, which is beneficial to improving the accuracy and reliability of the interpolated green component corresponding to each calculated pixel, and thus beneficial to ensuring the image visual effect.

[0129] In another optional embodiment, when the determined red component position is a partial red component position, the determined blue component position is a partial blue component position, and the determined green component position is a partial green component position, before the image signal processor performs a blue component interpolation operation on the pixel at the determined red component position and a red component interpolation operation on the pixel at the determined blue component position according to the interpolated full-resolution green channel, the method further includes:

[0130] The image signal processor determines, according to the determined remosaicking pattern, the specified red component position in the unit of the red component as the red component position where a blue component interpolation operation needs to be performed, and determines the specified blue component position in the unit of the blue component as the blue component position where a red component interpolation operation needs to be performed.

[0131] Wherein, before the image signal processor performs a red component interpolation operation and a blue component interpolation operation on the pixel at the determined green component position according to the interpolated blue channel and red channel, the method further includes:

[0132] The image signal processor determines, according to the remosaicking pattern, the specified first green component position in the unit of the green component as the green component position where a blue component interpolation operation needs to be performed and the specified second green component position in the unit of the green component as the green component position where a red component interpolation operation needs to be performed.

[0133] Optionally, the image signal processor combines all the interpolated color channels to obtain a remosaicked image, including:

[0134] The image signal processor combines all the interpolated color channels according to the remosaicking pattern to obtain a mosaic image in the target arrangement manner.

[0135] It can be seen that this optional embodiment can also determine the color channels that need to be interpolated and the arrangement mode of the demosaicked image obtained by subsequent combination according to the required demosaicking pattern, which is beneficial to improving the matching degree between the demosaicked image obtained by combination and the actual arrangement mode requirement, and further beneficial to improving the reliability of the image signal processor to perform the demosaicking operation. In addition, only the color components need to be interpolated in the color channels at the specified pixel positions, which is beneficial to saving the calculation cost and improving the efficiency of obtaining the demosaicked image.

[0136] In the embodiment of the present invention, an example is given by taking an image of a four-unit Bayer color filter array (Quad Bayer CFA) as an example. The specific implementation steps for demosaicking the four-unit Bayer color filter array are as follows:

[0137] 1) Input an image of a four-unit Bayer color filter array (Quad Bayer CFA). Among them, the first color channel corresponds to the red R component, the second color channel corresponds to the green G component, and the third color channel corresponds to the blue B component. Only one of the R, G, and B color components exists in each pixel point of this image, and each unit (2*2 pixel points) has the same color component.

[0138] 2) According to the unit where the current pixel point for green channel interpolation is located, calculate the gradients grad H 、grad V 、grad D 、grad A of this unit in four directions (horizontal direction, vertical direction, diagonal direction, anti-diagonal direction). G i,j represents a green pixel point, and RB i,j can correspond to a red or blue pixel point. Please refer to the CFA image of the blue pixel point and its unit located at the center of the window as shown in Figure 6 , and the CFA image of the red pixel point and its unit located at the center of the window as shown in Figure 7 . Among them, the calculation formulas of grad H 、grad V 、grad D 、grad A are as follows:

[0139] grad H = abs(G 32 - G 35 ) + abs(G 42 - G 45 ) + abs(G 23 - G 24 ) + abs(G 53 - G54 ) + abs(RB 33 -RB 34 ) + abs(RB 43 -RB 44 ) + [abs(RB 22 -RB 25 ) + abs(RB 52 -RB 55 )] / 2

[0140] grad V =abs(G 23 -G 53 ) + abs(G 24 -G 54 ) + abs(G 32 -G 42 ) + abs(G 35 -G 45 ) + abs(RB 33 -RB 43 ) + abs(RB 34 -RB 44 ) + [abs(RB 22 -RB 52 ) + abs(RB 25 -RB 55 )] / 2

[0141] grad D =abs(G 23 -G 45 ) + abs(G 32 -G 54 ) + abs(G 24 -G 35 ) + abs(G 42 -G 53 ) + abs(RB 33 -RB 34 )×2 + abs(RB 22 -RB 55 )

[0142] grad A =abs(G 24 -G 42 ) + abs(G 35 -G 53 ) + abs(G 23 -G 32 ) + abs(G 45 -G 54 ) + abs(RB 34 -RB 43 )×2 + abs(RB 25-RB 52 )

[0143] 3) Calculate its texture gradient matrix M0 based on the unit where the current pixel is located, and calculate the texture gradient matrices M1 to M8 of the surrounding units respectively. Specifically, this step of calculation can be carried out within a window of 6*6 size. The specific calculation process is as follows:

[0144] M0 = [(P0 11 -P0 12 ),(P0 11 -P0 21 ),(P0 11 -P0 22 ),(P0 12 -P0 21 ),(P0 22 -P0 12 ),(P0 22 -P0 21 ),(P0 11 +P0 12 -P0 21 -P0 22 ),(P0 11 +P0 21 -P0 12 -P0 22 ),(P0 11 +P0 22 -P0 12 -P0 21 )]

[0145] M1 = [(P1 11 -P1 12 ),(P1 11 -P1 21 ),(P1 11 -P1 22 ),(P1 12 -P1 21 ),(P1 22 -P1 12 ),(P1 22 -P1 21 ),(P1 11 +P1 12 -P1 21 -P1 22 ),(P1 11 +P1 21 -P1 12 -P1 22 ),(P1 11 +P1 22 -P1 12 -P1 21 )] ...

[0147] M8 = [(P8 11 - P8 12 ),(P8 11 - P8 21 ),(P8 11 - P8 22 ),(P8 12 - P8 21 ),(P8 22 - P8 12 ),(P8 22 - P8 21 ),(P8 11 + P8 12 - P8 21 - P8 22 ),(P8 11 + P8 21 - P8 12 - P8 22 ),(P8 11 + P8 22 - P8 12 - P8 21 )]

[0148] 4) Calculate the texture similarity coefficient. Perform SAD operation using the texture gradient matrix calculated in 3). It should be noted that SSD operation or other algorithms in the prior art can also be used to perform corresponding operations on the calculated texture gradient matrix. The specific calculation process is as follows:

[0149]

[0150] 5) Determine the interpolation direction. Synthesize the results of 2) and 4) to obtain the values corresponding to four directions (horizontal, vertical, diagonal, anti - diagonal). Among them, w11, w12, w21, w22, w31, w32, w41, w42 are adjustable weight coefficients. Finally, find the minimum value among the values corresponding to the four directions, and use the direction corresponding to this value as the final interpolation direction. Thresh required in the calculation process is an adjustable threshold. Among them, the calculation formulas for the values corresponding to the four directions (horizontal, vertical, diagonal, anti - diagonal) and the code for determining the interpolation direction are as follows:

[0151]

[0152]

[0153]

[0154] if (Value == ValueH &&Value + Thresh < min(Value V , Value D , Value A ))

[0155] direction = horizontal

[0156] elseif (Value == Value V &&Value + Thresh < min(Value H , Value D , Value A ))

[0157] direction = vertical

[0158] elseif (Value == Value D &&Value + Thresh < min(Value H , Value V , Value A ))

[0159] direction = diagonal

[0160] elseif (Value == Value A &&Value + Thresh < min(Value H , Value V , Value D ))

[0161] direction = anti - diagonal

[0162] else

[0163] direction = none

[0164] 6) Interpolate the G channel. According to the interpolation direction determined in 5), interpolate the missing green pixel values of each pixel in the unit. For example, perform green component interpolation on the positions of B33, B34, B43, B44 as shown in Figure 6 , and perform green component interpolation on the positions of R33, R34, R43, R44 as shown in Figure 7 . The specific process for determining the interpolated components is as follows:

[0165] if (direction == horizontal)

[0166]

[0167] else if (direction == vertical)

[0168]

[0169] else if (direction == diagonal)

[0170]

[0171] else if (direction == anti - diagonal)

[0172]

[0173] else if (directon == none)

[0174]

[0175] Among them, w1, w2, w3, and w4 are weight coefficients.

[0176] 7) Interpolate the B and R components for the R and B channels respectively. It can be obtained according to the full - resolution G channel by using the color correlation between G and the RB channels within a local range. The interpolation of the B and R components can be achieved using, but not limited to, methods such as color difference constancy or guided filtering.

[0177] For example: If a remosaicked image arranged in the RGGB form is finally required, for Figure 8 the case shown in 8a, interpolate the red component for the blue pixel B33 to obtain R33. For Figure 8 the case shown in 8b, interpolate the blue component for the red pixel R44 to obtain B44. That is, the results after interpolating the B and R components for the R and B channels respectively can be as shown in Figure 8 8a and 8b respectively.

[0178] 8) Interpolate the B and R components for the G channel, as shown in Figure 9 9a and 9b. It can be obtained according to the full - resolution G channel by using the color correlation between G and the RB channels within a local range. The interpolation of the B and R components for the G channel can be achieved using, but not limited to, methods such as color difference constancy or guided filtering.

[0179] For example: If a remosaicked image arranged in the RGGB form is finally required, for Figure 9In the cases shown in 9a and 9b, the red component of the green pixel G33 is interpolated to obtain R33, the blue component of the green pixel G44 is interpolated to obtain B44, and the results of interpolating the B and R components for the G channel can be respectively as shown in Figure 10 10a and 10b shown in

[0180] 9) Combine the R, G, and B channels to generate a remosaiced image in a certain arrangement, specifically as shown in Figure 11 shown

[0181] It should be noted that the method described in the embodiments of the present invention not only supports Quad Bayer CFA images, but also supports Quad RCCB CFA images, such as the Quad RGBW format image shown in 12a in Figure 12 and the Quad RGBIR format image shown in 12b in Figure 12 as well as, such as the Quad RGBW format image with a minimum sampling period of 4*4 shown in 13a in Figure 13 and the Quad RGBIR format image with a minimum sampling period of 4*4 shown in 13b in Figure 13 For the Quad RGBW format image, only the G channel with pixel W needs to be interpolated first, and the rest is similar to the Quad Bayer CFA image; for the Quad RGBIR format image, only the G channel with pixel IR needs to be interpolated first, and the rest is similar to the Quad Bayer CFA image; for the Quad RGBIR format image with a minimum sampling period of 4*4, only the R, G, and B channels with pixel IR need to be interpolated first, and the rest is similar to the Quad Bayer CFA image; for the Quad RGBW format image with a minimum sampling period of 4*4, all the W channels are interpolated first, then the G channels with pixels R / B are interpolated, so that the G channel with pixel W can be interpolated, and then using the relationship between W and the R, G, and B channels, the B / R channels with pixels R / B are interpolated, so that the R / B channels with pixels G can be interpolated, and finally the R and B channels with pixels W are interpolated to complete the remosaicing operation.

[0182] It should be noted that when the image signal processor processes a nine - unit color filter array image or a sixteen - unit color filter array image, it can downsample them into multiple groups of four - unit color filter array images in a certain way. For example, for a nine - unit color filter array image, every 9 units are downsampled into 4 groups, and for a sixteen - unit color filter array image, every 16 units are downsampled into 9 groups, which is better than the downsampling schemes of every 9 units being downsampled into 4 groups and every 16 units being downsampled into 16 groups in the prior art. Then, the remosaicking process is performed through the method described in the embodiments of the present invention, and finally, the images are fused to obtain a full - resolution RGB image.

[0183] Embodiment III

[0184] Please refer to Figure 14 , Figure 14 , which is a schematic structural diagram of an implementation device for remosaicking disclosed in the embodiments of the present invention. Among them, Figure 14 the described device can be applied to an image signal processor. As Figure 14 shown, the implementation device for remosaicking may include:

[0185] An image acquisition module 301, configured to acquire a multi - unit CFA map to be processed.

[0186] A remosaicking module 302, configured to calculate the gradients of the unit where the current pixel point that needs to perform green - channel interpolation in the multi - unit CFA image in multiple target directions, and calculate the texture gradient matrix of the unit where the current pixel point is located and the texture gradient matrix of each adjacent unit among all adjacent units of the unit where the current pixel point is located; and, calculate the texture similarity coefficient according to the texture gradient matrix of the unit where the current pixel point is located and the texture gradient matrix of each adjacent unit, determine the target interpolation direction of the unit where the current pixel point is located according to the texture similarity coefficient and the gradients of the unit where the current pixel point is located in each target direction, and perform a green - component interpolation operation on all pixel points of the unit where the current pixel point is located in the target interpolation direction; and, perform a blue - component interpolation operation on the pixel points at the determined red - component positions and perform a red - component interpolation operation on the pixel points at the determined blue - component positions according to the interpolated full - resolution green channel, and, perform a red - component interpolation operation and a blue - component interpolation operation on the pixel points at the determined green - component positions according to the interpolated blue channel and red channel.

[0187] A combination module 303, configured to combine all the interpolated color channels to obtain a remosaicked image.

[0188] Optionally, all target directions include the horizontal direction, the vertical direction, the diagonal direction, and the anti - diagonal direction.

[0189] It can be seen that in the implementationFigure 3 The described device can enable an image signal processor to compatibly process multi-unit CFA images. While restoring the missing parts in the multi-unit CFA images to ensure the visual effect of the images, it can reduce the algorithm complexity and hardware overhead, which is beneficial for hardware implementation, and the processing flow has real-time performance.

[0190] In an optional embodiment, the remosaicking module 302 is further configured to perform a pseudo-color removal operation on the remosaicked image to obtain a full-color image when the determined red component positions are all red component positions, the determined blue component positions are all blue component positions, and the determined green component positions are all green component positions.

[0191] It can be seen that this optional embodiment can also combine remosaicking and demosaicking, and perform pseudo-color detection and removal operations during the demosaicking stage, which is beneficial for greatly saving computational overhead.

[0192] Optionally, as Figure 14 shown, the device may further include:

[0193] An image output module 304, configured to convert the full-color image into an image in a target format and output the image in the target format after obtaining the full-color image.

[0194] It can be seen that this optional embodiment can also convert the full-color image into an image in a required format and output it after performing a remosaicking operation on the multi-unit CFA image and processing the remosaicked image to obtain the full-color image, which is beneficial for improving the matching degree between the output image and the actual requirements.

[0195] In another optional embodiment, as Figure 15 shown, the device may further include:

[0196] A demosaicking module 305, configured to perform a demosaicking operation on the remosaicked image to obtain a full-color image when the determined red component positions are partial red component positions, the determined blue component positions are partial blue component positions, and the determined green component positions are partial green component positions.

[0197] In this optional embodiment, as Figure 14 shown, the device may further include an image output module 304. At this time, the full-color image in the full-color image that the image output module 304 converts into an image in a target format is the full-color image obtained by the demosaicking module 305 performing a demosaicking operation.

[0198] It can be seen that this optional embodiment can also add a remosaicking module 302 before the demosaicking module 305, implementing the function of re - fusing a multi - unit CFA image into a single - unit CFA image that actually needs according to the arrangement format of the multi - unit CFA image.

[0199] Optionally, the specific way for the remosaicking module 302 to determine the target interpolation direction of the unit where the current pixel is located according to the texture similarity coefficient and the gradient of the unit where the current pixel is located in each target direction is as follows:

[0200] Calculate the value corresponding to each target direction according to the gradient of the unit where the current pixel is located in each of the target directions, the texture similarity coefficient, and the weight coefficient corresponding to each target direction;

[0201] Determine the target interpolation direction of the unit where the current pixel is located according to the target minimum value among the values corresponding to all target directions, the pre - determined threshold value, and the remaining values except the target minimum value among the values corresponding to all target directions.

[0202] Further optionally, the specific way for the remosaicking module 302 to determine the target interpolation direction of the unit where the current pixel is located according to the target minimum value among the values corresponding to all target directions, the pre - determined threshold value, and the remaining values except the target minimum value among the values corresponding to all target directions can be as follows:

[0203] Judge whether the sum of the target minimum value among the values corresponding to all target directions and the pre - determined threshold value is less than the minimum value among all the remaining values, where all the remaining values include the values corresponding to all target directions except the target minimum value;

[0204] When the judgment result is yes, determine the target direction corresponding to the target minimum value as the target interpolation direction of the unit where the current pixel is located;

[0205] When the judgment result is no, determine the empty direction as the target interpolation direction of the unit where the current pixel is located.

[0206] It can be seen that this optional embodiment can intelligently determine the interpolation direction of the unit where the current pixel is located according to the calculated value corresponding to each direction and the pre - determined threshold value to achieve green channel interpolation.

[0207] Still further optionally, the specific way for the remosaicking module 302 to perform the green - component interpolation operation on all pixels of the unit where the current pixel is located in the target interpolation direction is as follows:

[0208] Determine an interpolation green - component calculation formula that matches the target interpolation direction according to the target interpolation direction, and obtain the calculation parameters corresponding to the interpolation green - component calculation formula;

[0209] According to the interpolation green component calculation formula and the calculation parameters corresponding to the interpolation green component calculation formula, calculate the interpolation green component corresponding to each pixel in the unit where the current pixel is located, and perform the green component interpolation operation on this pixel in the target interpolation direction according to the interpolation green component corresponding to each pixel in the unit where the current pixel is located.

[0210] It can be seen that this optional embodiment can also intelligently match the corresponding interpolation green component calculation formula and calculation parameters according to the determined interpolation direction, and then calculate the interpolation green component corresponding to each pixel, which is beneficial to improving the accuracy and reliability of the interpolation green component corresponding to each calculated pixel, and further beneficial to ensuring the visual effect of the image.

[0211] Furthermore, optionally, the remosaicking module 302 is further configured to, when the determined red component position is a partial red component position, the determined blue component position is a partial blue component position, and the determined green component position is a partial green component position, and before performing the blue component interpolation operation on the pixel at the determined red component position and the red component interpolation operation on the pixel at the determined blue component position according to the interpolated full-resolution green channel, determine the specified red component position in the unit of the red component as the red component position that needs to perform the blue component interpolation operation, and determine the specified blue component position in the unit of the blue component as the blue component position that needs to perform the red component interpolation operation according to the determined remosaicking pattern.

[0212] Among them, the remosaicking module 302 is further configured to, when the determined red component position is a partial red component position, the determined blue component position is a partial blue component position, and the determined green component position is a partial green component position, and before performing the red component interpolation operation and the blue component interpolation operation on the pixel at the determined green component position according to the interpolated blue channel and red channel, determine the specified first green component position in the unit of the green component as the green component position that needs to perform the blue component interpolation operation and determine the specified second green component position in the unit of the green component as the green component position that needs to perform the red component interpolation operation according to the remosaicking pattern.

[0213] Furthermore, optionally, the specific manner in which the combining module 303 combines all the interpolated color channels to obtain the remosaicked image is as follows:

[0214] Combine all the interpolated color channels according to the remosaicking pattern to obtain a mosaic image in the target arrangement manner.

[0215] It can be seen that this alternative embodiment can also determine the color channels that need to be interpolated according to the required demosaicing pattern and the arrangement mode of the subsequent combined demosaiced image, which is beneficial to improving the matching degree between the combined demosaiced image and the actual arrangement mode requirements, and further beneficial to improving the reliability of the image signal processor in performing the demosaicing operation. In addition, only the color components need to be interpolated in the color channels at the specified pixel positions, which is beneficial to saving the calculation overhead and improving the efficiency of obtaining the demosaiced image.

[0216] Embodiment 4

[0217] Please refer to Figure 16 , Figure 16 which is a schematic structural diagram of another implementation device for demosaicing disclosed in the embodiments of the present invention. Among them, Figure 16 the described device can be applied to an image signal processor. As Figure 16 shown, the implementation device for demosaicing may include:

[0218] A memory 401 storing executable program code;

[0219] A processor 402 coupled to the memory 401;

[0220] The processor 402 calls the executable program code stored in the memory 401 and executes some or all of the steps in the implementation method for demosaicing disclosed in Embodiment 1 or Embodiment 2 of the present invention.

[0221] Embodiment 5

[0222] The embodiments of the present invention disclose a computer storage medium. The computer storage medium stores computer instructions, which are used to execute some or all of the steps in the implementation method for demosaicing disclosed in Embodiment 1 or Embodiment 2 of the present invention when the computer instructions are called.

[0223] The above-described device embodiments are merely illustrative. The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative labor.

[0224] Through the specific descriptions of the above embodiments, those skilled in the art can clearly understand that each implementation manner can be realized by means of software plus a necessary general hardware platform, and of course, it can also be realized by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, and the storage medium includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc memories, magnetic disk memories, tape memories, or any other medium that can be used to carry or store data and is computer-readable.

[0225] Finally, it should be noted that: The implementation method and device for re-mosaicing disclosed in the embodiments of the present invention only disclose the preferred embodiments of the present invention, and are only used to illustrate the technical solutions of the present invention, rather than limiting them; Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; And these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for implementing heavy mosaic, characterized in that, The method includes: Obtaining a multi-unit CFA image to be processed, calculating gradients of the unit where the current pixel point that needs to perform green channel interpolation in the multi-unit CFA image in multiple target directions, calculating a texture gradient matrix of the unit where the current pixel point is located, and texture gradient matrices of each of all adjacent units of the unit where the current pixel point is located; Calculating a texture similarity coefficient according to the texture gradient matrix of the unit where the current pixel point is located and the texture gradient matrices of each of the adjacent units, determining a target interpolation direction of the unit where the current pixel point is located according to the texture similarity coefficient and the gradients of the unit where the current pixel point is located in each of the target directions, and performing a green component interpolation operation on all pixel points of the unit where the current pixel point is located in the target interpolation direction; Performing a blue component interpolation operation on the pixel points at the determined red component positions and performing a red component interpolation operation on the pixel points at the determined blue component positions according to the interpolated full-resolution green channel, and performing a red component interpolation operation and a blue component interpolation operation on the pixel points at the determined green component positions according to the interpolated blue channel and red channel; Combining all the color channels obtained by interpolation to obtain a remosaicked image; Among them, the determining the target interpolation direction of the unit where the current pixel point is located according to the texture similarity coefficient and the gradients of the unit where the current pixel point is located in each of the target directions includes: Calculating a value corresponding to each of the target directions according to the gradients of the unit where the current pixel point is located in each of the target directions, the texture similarity coefficient, and the weight coefficient corresponding to each of the target directions; Determining the target interpolation direction of the unit where the current pixel point is located according to the target minimum value among all the values corresponding to the target directions, a pre-determined threshold, and the remaining values among all the values corresponding to the target directions except the target minimum value.

2. The method for realizing remosaicing according to claim 1, wherein The method further includes: When the determined red component positions are partial red component positions, the determined blue component positions are partial blue component positions, and the determined green component positions are partial green component positions, performing a demosaicking operation on the remosaicked image to obtain a full-color image; When the determined red component positions are all red component positions, the determined blue component positions are all blue component positions, and the determined green component positions are all green component positions, performing a pseudo-color removal operation on the remosaicked image to obtain a full-color image.

3. The implementation method of remosaicking according to claim 2, characterized in that, The method further includes: After obtaining the full-color image, converting the full-color image into an image in a target format and outputting the image in the target format; Among them, all the target directions include a horizontal direction, a vertical direction, a diagonal direction, and an anti-diagonal direction.

4. The method for implementing re-mosaicing according to claim 1, wherein The determining the target interpolation direction of the unit where the current pixel point is located according to the target minimum value among all the values corresponding to the target directions, a pre-determined threshold, and the remaining values among all the values corresponding to the target directions except the target minimum value includes: Determine whether the sum of the target minimum value among the values corresponding to all the target directions and a previously determined threshold is less than the minimum value among all the remaining values, where all the remaining values include the values corresponding to all the target directions except the target minimum value; When the determination result is yes, determine the target direction corresponding to the target minimum value as the target interpolation direction of the unit where the current pixel point is located; When the determination result is no, determine the empty direction as the target interpolation direction of the unit where the current pixel point is located.

5. The method for implementing re-mosaicking according to claim 4, characterized in that, The performing a green component interpolation operation on all pixel points of the unit where the current pixel point is located in the target interpolation direction includes: According to the target interpolation direction, determine an interpolation green component calculation formula that matches the target interpolation direction, and obtain calculation parameters corresponding to the interpolation green component calculation formula; According to the interpolation green component calculation formula and the calculation parameters corresponding to the interpolation green component calculation formula, calculate the interpolation green component corresponding to each pixel point in the unit where the current pixel point is located, and perform a green component interpolation operation on the pixel point in the target interpolation direction according to the interpolation green component corresponding to each pixel point in the unit where the current pixel point is located.

6. The method for implementing remosaicking according to any one of claims 2-5, characterized in that When the determined red component position is a partial red component position, the determined blue component position is a partial blue component position, and the determined green component position is a partial green component position, before performing a blue component interpolation operation on the pixel points at the determined red component position and a red component interpolation operation on the pixel points at the determined blue component position according to the interpolated full-resolution green channel, the method further includes: According to the determined remosaicking pattern, determine the specified red component position in the unit of the red component as the red component position that needs to perform a blue component interpolation operation, and determine the specified blue component position in the unit of the blue component as the blue component position that needs to perform a red component interpolation operation; And, before performing a red component interpolation operation and a blue component interpolation operation on the pixel points at the determined green component position according to the interpolated blue channel and red channel, the method further includes: According to the remosaicking pattern, determine the specified first green component position in the unit of the green component as the green component position that needs to perform a blue component interpolation operation and determine the specified second green component position in the unit of the green component as the green component position that needs to perform a red component interpolation operation; Wherein, the combining all the color channels obtained by interpolation to obtain a remosaicked image includes: Combine all the color channels obtained by interpolation according to the remosaicking pattern to obtain a mosaic image in a target arrangement manner.

7. An apparatus for implementing heavy mosaic, characterized in that, The device is used to execute the remosaicking implementation method according to any one of claims 1-6, and the device includes: An image acquisition module, configured to acquire a multi-unit CFA image to be processed; A remosaicking module, which is used to calculate the gradients of the unit where the current pixel point that needs to perform green channel interpolation in the multi-unit CFA image in multiple target directions, calculate the texture gradient matrix of the unit where the current pixel point is located, and the texture gradient matrix of each adjacent unit among all adjacent units of the unit where the current pixel point is located; and, calculate the texture similarity coefficient according to the texture gradient matrix of the unit where the current pixel point is located and the texture gradient matrix of each adjacent unit, determine the target interpolation direction of the unit where the current pixel point is located according to the texture similarity coefficient and the gradients of the unit where the current pixel point is located in each of the target directions, and perform green component interpolation operations on all pixel points of the unit where the current pixel point is located in the target interpolation direction; and, perform blue component interpolation operations on the pixel points at the determined red component positions and perform red component interpolation operations on the pixel points at the determined blue component positions according to the interpolated full-resolution green channel, and, perform red component interpolation operations and blue component interpolation operations on the pixel points at the determined green component positions according to the interpolated blue channel and red channel; A combination module, which is used to combine all the color channels obtained by interpolation to obtain a remosaicked image.

8. The implementation device for re-mosaicking according to claim 7, characterized in that, The device further includes: A demosaicking module, which is used to perform demosaicking processing operations on the remosaicked image to obtain a full-color image when the determined red component positions are partial red component positions, the determined blue component positions are partial blue component positions, and the determined green component positions are partial green component positions.

9. The implementation device for re-mosaicing according to claim 7, characterized in that, The remosaicking module is further used to perform pseudo-color removal operations on the remosaicked image to obtain a full-color image when the determined red component positions are all red component positions, the determined blue component positions are all blue component positions, and the determined green component positions are all green component positions.

10. An apparatus for implementing heavy mosaic, characterized in that, The device includes: A memory storing executable program code; A processor coupled to the memory; The processor calls the executable program code stored in the memory and executes the implementation method of remosaicking according to any one of claims 1-6.

11. A computer storage medium, characterized in that, The computer storage medium stores computer instructions, which are used to execute the implementation method of remosaicking according to any one of claims 1-6 when being called.

Images