Local chromatic aberration self-adaptive image demosaicing interpolation method, system-level chip and computer storage medium

Abstract

The invention provides a local color difference self-adaptive image demosaicing interpolation method, a system-level chip and a computer storage medium. The method comprises the following steps: acquiring an original RAW image; taking a current target pixel in the RAW image as a reference, and extracting a cache window; in the cache window, gradient values of the current target pixel in the horizontal direction, the vertical direction, the diagonal direction and the reverse diagonal direction are calculated; determining an interpolation direction according to the gradient value; and along the interpolation direction, carrying out RGB full-channel interpolation calculation. In the interpolation process, according to the difference between the center point and the two ends of the main direction, the weights of the pixel points participating in interpolation are obtained in a self-adaptive mode, the interpolation precision can be guaranteed, meanwhile, the size of a calculation window is reduced, the resolution of the image can be effectively improved in the interpolation calculation process, the quality of the demosaiced image is improved, and the image quality is improved. The consumption of hardware resources can be saved, and the problem that the image quality is poor due to the fact that an interpolation method adopted in an existing demosaicing process is poor in precision and high in hardware consumption is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a local color difference adaptive image de-mosaicing interpolation method, a system-level chip, and a computer storage medium. Background Art

[0002] Current imaging devices, such as cameras, usually use a single image sensor with a filter for cost considerations. After the image sensor captures the image, the ISP (Image Signal Processing) system is used to process the captured image, and finally outputs an RGB color image. ISP usually includes linear error correction, denoising, bad pixel removal, white balance, color correction, tone mapping, demosaicing, etc.

[0003] Among them, the demosaicing process is to convert the image from the RAW domain (the original image data output by the image sensor) to the RGB domain (the image with color) by interpolation according to the gradient direction. In the existing demosaicing process, the gradients in eight directions (E east, N north, W west, S south, EN northeast, WN northwest, WS southwest, ES southeast) are usually calculated during interpolation to confirm the interpolated gradient direction, and then the color difference estimation value of the current target pixel is calculated from the selected gradient direction, and then the RGB value of the current target pixel is obtained.

[0004] However, the interpolation method used in the existing de-mosaicing process selects 8 directions, but there are too few pixels in each direction, so a larger window is needed to increase the number of pixels to ensure the accuracy of gradient calculation; but since the spatial relationship of pixel values ​​will cause the confidence to decrease with the increase of distance, when the window is increased, the confidence of the pixel value farther away from the current target pixel is lower, which affects the accuracy of gradient calculation to a certain extent; at the same time, the calculation process of a large window is bound to increase hardware consumption, resulting in higher hardware costs.

[0005] In addition, in the existing de-mosaicing process, due to the influence of the accuracy of interpolation calculation, problems such as false color, aliasing effect, moiré and purple fringing will occur, which seriously affect the image quality. For example, during the interpolation calculation, if the local pixel value is averaged incorrectly or the image does not overlap as expected, false color will appear; in an area with a particularly large color span of an image or at the edge where two scenes intersect, when processing the data, the pixel value with a relatively smooth and similar surrounding color is not selected, but the color interpolation calculation is continued along the sudden change area. If a bilinear interpolation algorithm is used, it is very easy to cause a aliasing effect; for dense texture areas in the image, moiré will appear after interpolation; when the interpolation of red and blue pixel values ​​is aliased, purple fringing will appear. Summary of the invention

[0006] The purpose of the present invention is to provide a local color difference adaptive image demosaicing interpolation method and a system-level chip, and a computer storage medium to solve the problem that the interpolation method used in the existing demosaicing process has poor accuracy and high hardware consumption, resulting in poor image quality.

[0007] In order to solve the above technical problems, the present invention provides an image demosaicing interpolation method with local color difference self-adaptation, comprising:

[0008] Get the original RAW image;

[0009] Extract the cache window based on the current target pixel in the RAW image;

[0010] In the cache window, calculate the gradient value of the current target pixel in the horizontal direction, vertical direction, diagonal direction and anti-diagonal direction;

[0011] Determine the interpolation direction according to the gradient value in the horizontal direction, the gradient value in the vertical direction, the gradient value in the diagonal direction, and the gradient value in the anti-diagonal direction;

[0012] Along the interpolation direction, RGB full channel interpolation calculation is performed to output the demosaiced image.

[0013] Optionally, in the local color difference adaptive image demosaicing interpolation method, the step of extracting the cache window based on the current target pixel in the RAW image includes:

[0014] Taking the current target pixel in the RAW image as a reference, extract a cache window of m rows and n columns, where m is an odd number greater than or equal to 5, and n is an odd number greater than or equal to 5;

[0015] The pixel value of the pixel in the cache window is recorded as P x,y , where x represents the row coordinate of the pixel in the cache window, in the order from left to right, x = 0, 1, ..., m-1; y represents the column coordinate of the pixel in the cache window, in the order from top to bottom, y = 0, 1, ..., n-1;

[0016] The pixel value of the current target pixel is recorded as P i,j ,in,

[0017] Optionally, in the local color difference adaptive image demosaic interpolation method, the step of calculating the gradient value of the current target pixel in the horizontal direction, the vertical direction, the diagonal direction and the anti-diagonal direction in the cache window includes:

[0018] In the cache window, calculate the same-channel pixel difference values ​​of the current target pixel in the horizontal direction, vertical direction, diagonal direction and anti-diagonal direction;

[0019] In the cache window, calculate the difference values ​​of adjacent pixels of the current target pixel in the horizontal direction, vertical direction, diagonal direction and anti-diagonal direction;

[0020] In the cache window, calculate the color difference weight of the current target pixel;

[0021] The gradient value in the horizontal direction is calculated using the color difference weight, the difference value of the same-channel pixels in the horizontal direction, and the difference value of the adjacent pixels in the horizontal direction; the gradient value in the vertical direction is calculated using the color difference weight, the difference value of the same-channel pixels in the vertical direction, and the difference value of the adjacent pixels in the vertical direction; the gradient value in the diagonal direction is calculated using the color difference weight, the difference value of the same-channel pixels in the diagonal direction, and the difference value of the adjacent pixels in the diagonal direction; the gradient value in the anti-diagonal direction is calculated using the color difference weight, the difference value of the same-channel pixels in the anti-diagonal direction, and the difference value of the adjacent pixels in the anti-diagonal direction.

[0022] Optionally, in the local color difference adaptive image demosaicing interpolation method, the step of calculating the same-channel pixel difference values ​​of the current target pixel in the horizontal direction, the vertical direction, the diagonal direction and the anti-diagonal direction in the cache window includes:

[0023] Calculate the pixel difference value channel_diff_h of the same channel of the current target pixel in the horizontal direction, where the calculation formula is:

[0024] channel_diff_h

[0025] =(|P i-2，j-2 -P i-2，j |+|P i-2，j -P i-2，j+2 |+2×|P i，j-2 -P i，j |+2

[0026] ×|P i，j- P i，j+2 |+|P i+2，j-2 -P i+2，j |+|P i+2，j -P i+2，j+2 |+4

[0027] ×|P i-1，j-1 -P i-1，j+1 |+4×|P i+1，j-1 -P i+1，j+1 |+2

[0028] ×|Pi-2，j-1 -P i-2，j+1 |+2×|P i+2，j-1 -P i+2，j+1 |+4×|P i，j-1 -P i，j+1 |

[0029] +2×|P i-1，j-2 -P i-1，j |+2×|P i-1，j -P i-1，j+2 |+2×|P i+1，j-2 -P i+1，j |

[0030] +2×|P i+1，j -P i+1，j+2 |) / 32;

[0031] Calculate the same channel pixel difference value channel_diff_v in the vertical direction of the current target pixel, where the calculation formula is:

[0032] channel_diff_v

[0033] =(|P i-2，j-2 -P i，j-2 |+|P i，j-2 -P i+2，j-2 |+2×|P i-2，j -P i，j |+2

[0034] ×|P i，j -P i+2，j |+|P i-2，j+2 -P i，j+2 |+|P i，j+2 -P i+2，j+2 |+4

[0035] ×|P i-1，j-1 -P i+1，j-1 |+4×|P i-1，j+1 -P i+1，j+1 |+2

[0036] ×|P i-1，j-2 -P i+1，j-2 |+2×|P i-1，j+2 -P i+1，j+2 |+4×|P i-1，j -P i+1，j |

[0037] +2×|P i-2，j-1 -P i，j-1 |+2×|P i，j-1 -P i+2，j-1 |+2×|P i-2，j+1 -P i，j+1 |

[0038] +2×|P i，j+1 -P i+2，j+1 |) / 32;

[0039] Calculate the same channel pixel difference value channel_diff_d of the current target pixel in the diagonal direction, where the calculation formula is:

[0040] channel_diff_d

[0041] =(|P i-2，j -P i，j-2 |+|P i-2，j+2 -P i，j |+|P i+2，j-2 -P i，j |+|P i，j+2 -P i+2，j |

[0042] +2×(|P i-2，j+1 -P i，j-1 |+|P i-1，j -P i+1，j-2 |+|P i-1，j+2 -P i+1，j |

[0043] +|P i，j+1 -P i+2，j-1 |)+4×|P i-1，j+1 -P i+1，j-1 |) / 16;

[0044] Calculate the same-channel pixel difference value channel_diff_ad of the current target pixel in the anti-diagonal direction, where the calculation formula is:

[0045] channel_diff_ad

[0046] =(|P i-2，j -P i，j+2 |+|P i-2，j-2 -P i，j |+|P i+2，j+2 -P i，j |+|P i，j-2 -P i+2，j |

[0047] +2×(|P i-2，j-1 -P i，j+1 |+|P i-1，j -P i+1，j+2 |+|P i-1，j-2 -P i+1，j |

[0048] +|P i，j-1 -Pi+2，j+1 |)+4×|P i-1，j-1 -P i+1，j+1 |) / 16.

[0049] Optionally, in the local color difference adaptive image demosaicing interpolation method, the step of calculating the difference values ​​of adjacent pixels of the current target pixel in the horizontal direction, the vertical direction, the diagonal direction and the anti-diagonal direction in the cache window includes:

[0050] Calculate the horizontal difference value color_diff_h of the adjacent pixels of the current target pixel, where the calculation formula is:

[0051] color_diff_h=(|P i-2，j-2 -P i-2，j-1 |+|P i-2，j-1 -P i-2，j |+|P i-2，j -P i-*，j+1 |

[0052] +|P i-2，j+1 -P i-2，j+2 |+|P i+2，j-2 -P i+2，j-1 |+|P i+2，j-1 -P i+2，j |

[0053] +|P i+2，j -P i+2，j+1 |+|P i+2，j+1 -P i+2，j+2 |+2

[0054] ×(|P i-1，j-2 -P i-1，j-1 |+|P i-1，j-1 -P i-1，j |+|P i-1，j -P i-1，j+1 |

[0055] +|P i-1，j+1 -P i-1，j+2 |+|P i，j-2 -P i，j-1 |+|P i，j-1 -P i，j |+|P i，j -P i，j+1 |

[0056] +|P i，j+1 -P i，j+2 |+|P i+1，j-2 -P i+1，j-1 |+|P i+1，j-1 -P i+1，j |

[0057] +|P i+1，j -P i+1，j+1 |+|P i+1，j+1 -P i+1，j+2 |)) / 32;

[0058] Calculate the vertical difference value color_diff_v of the adjacent pixels of the current target pixel, where the calculation formula is:

[0059] color_diff_v=(|P i-2，j-2 -P i-1，j-2 |+|P i-1，j-2 -P i，j-2 |+|P i，j-2 -P i+1，j-2 |

[0060] +|P i+1，j-2 -P i+2，j-2 |+|P i-2，j+2 -P i-1，j+2 |+|P i-1，j+2 -P i，j+2 |

[0061] +|P i，j+2 -P i+1，j+2 |+|P i+1，j+2 -P i+2，j+2 |+2

[0062] ×(|P i-2，j-1 -P i-1，j-1 |+|P i-1，j-1 -P i，j-1 |+|P i，j-1 -P i+1，j-1 |

[0063] +|P i+1，j-1 -P i+2，j-1 |+|P i-2，j -P i-1，j |+|P i-1，j -P i，j |+|P i，j -P i+1，j |

[0064] +|P i+1，j -P i+2，j |+|P i-2，j+1 -P i-1，j+1 |+|P i-1，j+1 -P i，j+1 |

[0065] +|P i，j+1 -P i+1，j+1 |+|P i+1，j+1 -P i+2，j+1 |)) / 32;

[0066] Calculate the adjacent pixel difference value color_diff_d of the current target pixel in the diagonal direction, where the calculation formula is:

[0067] color_diff_d=(|P i-1，j-2 -P i-2，j-1 |+|P i，j-2 -P i-1，j-1 |+|P i-1，j-1 -P i-2，j |

[0068] +|P i+1，j-2 -P i，j-1 |+|P i，j-1 -P i-1，j |+|P i-1，j -P i-2，j+1 |

[0069] +|P i+2，j-2 -P i+1，j-1 |+|P i+1，j-1 -P i，j |+|P i，j -P i-1，j+1 |

[0070] +|P i-1，j+1 -P i-2，j+2 |+|P i+2，j-1 -P i+1，j |+|P i+1，j -P i，j+1 |

[0071] +|P i，j+1 -P i-1，j+2 |+|P i+2，j -P i+1，j+1 |+|P i+1，j+1 -P i，j+2 |

[0072] +|P i+2，j+1 -P i+1，j+2 |) / 16;

[0073] Calculate the adjacent pixel difference value color_diff_ad of the current target pixel in the anti-diagonal direction, where the calculation formula is:

[0074] color_diffad

[0075] =(|P i-2，j+1 -P i-1，j+2 |+|P i-2，j -P i-1，j+1 |+|P i-1，j+1 -P i，j+2 |

[0076] +|P i-2，j-1 -P i-1，j |+|P i-1，j -P i，j+1 |+|P i，j+1 -P i+1，j+2 |

[0077] +|P i-2，j-2 -P i-1，j-1 |+|P i-1，j-1 -P i，j |+|P i，j -P i+1，j+1 |

[0078] +|P i+1，j+1 -P i+2，j+2 |+|P i-1，j-2 -P i，j-1 |+|P i，j-1 -P i+1，j |

[0079] +|P i+1，j -P i+2，j+1 |+|P i，j-2 -P i+1，j-1 |+|P i+1，j-1 -P i+2，j |

[0080] +|P i+1，j-2 -P i+2，j+1 |) / 16.

[0081] Optionally, in the local color difference adaptive image demosaic interpolation method, the step of calculating the color difference weight of the current target pixel in the cache window includes:

[0082] Calculate the two adjacent color difference values ​​of the current target pixel in the horizontal direction and the vertical direction;

[0083] The larger value of the two adjacent color difference values ​​in the horizontal direction is taken as the color difference value in the horizontal direction, and the larger value of the two adjacent color difference values ​​in the vertical direction is taken as the color difference value in the vertical direction;

[0084] The smaller value of the color difference value in the horizontal direction and the color difference value in the vertical direction is taken as the local color difference value;

[0085] According to the set weight range, the local color difference value is range mapped to obtain the color difference weight.

[0086] Optionally, in the local color difference adaptive image demosaicing interpolation method, the color difference weight is denoted as color wt, 0≤color wt≤1;

[0087] The formula for calculating the gradient value grad_h in the horizontal direction is:

[0088] grad_h=(1-color_wt)×channel_diff_h+color_wt×color_diff_h;

[0089] The formula for calculating the gradient value grad_v in the vertical direction is:

[0090] grad_v=(1-color_wt)×channel_diff_v+color_wt×color_diff_v;

[0091] The formula for calculating the gradient value grad_d in the diagonal direction is:

[0092] grad_d=(1-color_wt)×channel_diff_d+color_wt×color_diff_d;

[0093] The formula for calculating the gradient value grad_ad in the anti-diagonal direction is:

[0094] grad_ad=(1-color_wt)×channel_diff_ad+color_wt×color_diff_ad.

[0095] Optionally, in the local chromatic aberration adaptive image demosaic interpolation method, the step of determining the interpolation direction according to the gradient value in the horizontal direction, the gradient value in the vertical direction, the gradient value in the diagonal direction, and the gradient value in the anti-diagonal direction comprises:

[0096] Set the undirected gradient threshold;

[0097] If the horizontal gradient value, the vertical gradient value, the diagonal gradient value and the anti-diagonal gradient value of the current target pixel are all less than the non-directional gradient threshold, the current target pixel is judged to be a non-directional interpolation point;

[0098] Otherwise, the interpolation direction is determined according to the gradient value in the horizontal direction, the gradient value in the vertical direction, the gradient value in the diagonal direction, and the gradient value in the anti-diagonal direction.

[0099] Optionally, in the local chromatic aberration adaptive image demosaic interpolation method, the step of determining the interpolation direction according to the gradient value in the horizontal direction, the gradient value in the vertical direction, the gradient value in the diagonal direction, and the gradient value in the anti-diagonal direction comprises:

[0100] The gradient values ​​in the horizontal direction and the gradient values ​​in the vertical direction are grouped together, and the gradient values ​​in the diagonal direction and the gradient values ​​in the anti-diagonal direction are grouped together;

[0101] Calculate the ratio of the smaller gradient value to the larger gradient value in each group, and take the group with the smaller ratio as the candidate direction group;

[0102] The direction corresponding to the smaller gradient value is selected from the direction candidate group as the interpolation direction.

[0103] Optionally, in the local color difference adaptive image demosaicing interpolation method, the step of performing RGB full channel interpolation calculation along the interpolation direction includes:

[0104] In the interpolation direction, extract the same-color pixels on both sides of the current target pixel;

[0105] The interpolation weights on both sides of the interpolation direction are calculated using the difference between two pixels of the same color and the current target pixel;

[0106] The channel value of the color channel to be interpolated of the current target pixel is calculated according to the interpolation weight.

[0107] Optionally, in the local color difference adaptive image demosaic interpolation method, the step of calculating the interpolation weights on both sides of the interpolation direction using the difference relationship between two same-color pixels and the current target pixel includes:

[0108] The pixel values ​​of two pixels of the same color are P a,b and P c,d , where the values ​​of a and c are or The values ​​of b and d are or

[0109] Calculate the color difference on both sides of the interpolation direction of the current target pixel, where the calculation formula is:

[0110] Diff_1=|P i,j -P a,b |

[0111] Diff_2=|P i,j -P c,d |

[0112] Diff=|Diff_1-Diff_2|

[0113] Where Diff_1 is the difference value of the same color pixels on one side of the interpolation direction, Diff_2 is the difference value of the same color pixels on the other side of the interpolation direction, and Diff is the difference value of the same color pixels on both sides of the interpolation direction of the current target pixel;

[0114] According to the set endpoint threshold value range, the same color difference values ​​on both sides of the interpolation direction of the current target pixel are range mapped to obtain the interpolation weight wt, where 0≤wt≤1.

[0115] Optionally, in the local color difference adaptive image demosaic interpolation method, the step of calculating the channel value of the to-be-interpolated color channel of the current target pixel according to the interpolation weight includes:

[0116] Calculate the color difference of the current target pixel on both sides of the interpolation direction, where the calculation formula is:

[0117]

[0118] Wherein, Cr_1 is the color difference of one side of the current target pixel in the interpolation direction, Cr_2 is the color difference of the other side of the current target pixel in the interpolation direction, and Cr is the composite color difference of the current target pixel;

[0119] Calculate the channel pixel value of the color channel to be interpolated of the current target pixel, where the calculation formula is:

[0120] Pixel C =P i,j -Cr.

[0121] In order to solve the above technical problems, the present invention also provides a system-level chip, which includes a processor and a memory, wherein the processor and the memory are coupled, and the processor is used to execute a computer program stored in the memory to execute the local color difference adaptive image de-mosaicing interpolation method as described in any of the above items.

[0122] In order to solve the above technical problems, the present invention also provides a computer storage medium, wherein the computer storage medium stores an executable program; when the executable program is executed, the local color difference adaptive image demosaicing interpolation method as described in any one of the above items is implemented.

[0123] The local chromatic aberration adaptive image demosaicing interpolation method, system-level chip, and computer storage medium provided by the present invention can adaptively obtain the weights of the pixel points involved in the interpolation according to the difference between the center point and the two ends of the main direction during the interpolation process, thereby reducing the size of the calculation window while ensuring the interpolation accuracy. Therefore, during the interpolation calculation, not only can the resolution of the image be effectively improved, the quality of the demosaiced image be improved, but also the consumption of hardware resources can be saved, thereby solving the problem that the interpolation method used in the existing demosaicing process has poor accuracy and high hardware consumption, resulting in poor image quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0124] Figure 1A flowchart of the local color difference adaptive image demosaicing interpolation method provided in this embodiment;

[0125] Figure 2 A schematic diagram of pixel arrangement in a 5×5 cache window provided in this embodiment where the current target pixel is R;

[0126] Figure 3 A flowchart of step S3 provided for this embodiment;

[0127] Figure 4 A flowchart of step S4 provided for this embodiment;

[0128] Figure 5 A flowchart of step S5 provided for this embodiment;

[0129] Figure 6 This is a schematic diagram of the structure of the system-on-chip provided in this embodiment. DETAILED DESCRIPTION

[0130] The following is a further detailed description of the local chromatic aberration adaptive image de-mosaic interpolation method, system-level chip, and computer storage medium proposed in the present invention in conjunction with the accompanying drawings and specific embodiments. It should be noted that the accompanying drawings are all in a very simplified form and use non-precise proportions, which are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention. In addition, the structure shown in the accompanying drawings is often a part of the actual structure. In particular, the emphasis that each drawing needs to show is different, and sometimes different proportions are used.

[0131] It should be noted that the terms "first", "second", etc. in the specification, claims and drawings of the present invention are used to distinguish similar objects in order to describe the embodiments of the present invention, rather than to describe a specific order or sequence. It should be understood that the structures used in this way can be interchanged under appropriate circumstances. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0132] This embodiment provides a partial color difference adaptive image demosaic interpolation method, such as Figure 1 As shown, including:

[0133] S1, obtaining the original RAW image;

[0134] S2, extracting the cache window based on the current target pixel in the RAW image;

[0135] S3, in the cache window, calculating the gradient values ​​of the current target pixel in the horizontal direction, vertical direction, diagonal direction and anti-diagonal direction;

[0136] S4, determining an interpolation direction according to a gradient value in a horizontal direction, a gradient value in a vertical direction, a gradient value in a diagonal direction, and a gradient value in an anti-diagonal direction;

[0137] S5, performing RGB full channel interpolation calculation along the interpolation direction to output a demosaiced image.

[0138] The local chromatic aberration adaptive image demosaicing interpolation method provided in the present embodiment can reduce the size of the calculation window while ensuring the interpolation accuracy by adaptively obtaining the weights of the pixel points participating in the interpolation according to the difference between the center point and the two ends of the main direction during the interpolation process. Therefore, during the interpolation calculation, not only can the resolution of the image be effectively improved, the quality of the image after demosaicing is improved, but also the consumption of hardware resources can be saved, thereby solving the problem that the interpolation method used in the existing demosaicing process has poor accuracy and high hardware consumption, resulting in poor image quality.

[0139] Specifically, in this embodiment, in step S1, the method for obtaining the original RAW image may be to obtain the original image from various types of photographic and video equipment.

[0140] RAW images are original image files obtained from image sensors of devices such as digital cameras, scanners, or film scanners. RAW images are not processed, so they have internal colors with a wide color gamut and can be adjusted accurately. In addition, in RAW images, the arrangement of RGB pixels (Bayer colors) has certain regularity, such as Figure 2 The figure shows the arrangement diagram of RGB pixels in the RGGB Bayer format, wherein the Gs in each row are arranged at intervals, and the Gs in adjacent rows are arranged in an alternating manner; adjacent Gs in the same row are all separated by R or B, and the R or B in adjacent rows is different, that is, when the Gs in the i-th row are all separated by R, the Gs in the i-1th row and the i+1th row are all separated by B; the arrangement rule of the columns is the same as that of the rows, which will not be repeated.

[0141] Further, in this embodiment, step S2, taking the current target pixel in the RAW image as a reference, extracting the cache window, specifically can be: taking the current target pixel in the RAW image as a reference, extracting a cache window of m rows and n columns, wherein m is an odd number greater than or equal to 5, and n is an odd number greater than or equal to 5; the pixel value of the pixel in the cache window is recorded as P x,y, where x represents the row coordinate of the pixel in the cache window, in the order from left to right, x = 0, 1, ..., m-1; y represents the column coordinate of the pixel in the cache window, in the order from top to bottom, y = 0, 1, ..., n-1; the pixel value of the current target pixel is recorded as P i,j ,in, That is, the coordinates of the pixels in the cache window can be expressed as (x, y), and the corresponding pixel value is expressed as P x,y ; The coordinates of the current target pixel can be expressed as (i, j), and its corresponding pixel value is recorded as P i,j .

[0142] In a specific embodiment, if Figure 2 As shown, a cache window of 5 rows and 5 columns can be extracted, that is, m=5, n=5. At this time, the current target pixel is located in the third row and third column, that is, i=2, j=2, and the pixel value of the current target pixel is recorded as P 2，2 .

[0143] exist Figure 2 In the buffer window shown, the RAW image is a Bayer image of RGGB, and the current target pixel is R, that is, it is necessary to perform interpolation calculations of G and B on the current target pixel. Of course, in practical applications, the RAW image may also be a Bayer image of other formats. Those skilled in the art can obtain a local color difference adaptive image demosaic interpolation method under a Bayer image of other formats according to the local color difference adaptive image demosaic interpolation method provided in this embodiment.

[0144] Of course, in practical applications, the size of the cache window can be reasonably selected according to the size and quality of the RAW image, and the number of rows and columns of the cache window does not need to be completely consistent, that is, the cache window can be square or rectangular. However, in order to ensure that there are enough pixels in the cache window for interpolation calculation, the number of rows and columns of the selected cache window must not be less than 5.

[0145] Furthermore, in this embodiment, step S3, in the cache window, calculates the gradient value of the current target pixel in the horizontal direction, vertical direction, diagonal direction and anti-diagonal direction, such as Figure 3 As shown, including:

[0146] S31, in the cache window, calculating the same-channel pixel difference values ​​of the current target pixel in the horizontal direction, the vertical direction, the diagonal direction and the anti-diagonal direction.

[0147] Specifically, in this embodiment, the calculation method of the same-channel pixel difference value of the current target pixel in each direction is given:

[0148] Calculate the pixel difference value channel_diff_h of the same channel of the current target pixel in the horizontal direction, where the calculation formula is:

[0149] channel_diff_h

[0150] =(|P i-2，j-2 -P i-2，j |+|P i-2，j -P i-2，j+2 |+2×|P i，j-2 -P i，j |+2

[0151] ×|P i，j -P i，j+2 |+|P i+2，j-2 -P i+2，j |+|P i+2，j -P i+2，j+2 |+4

[0152] ×|P i-1，j-1 -P i-1，j+1 |+4×|P i+1，j-1 -P i+1，j+1 |+2

[0153] ×|P i-2，j-1 -P i-2，j+1 |+2×|P i+2，j-1 -P i+2，j+1 |+4×|P i，j-1 -P i，j+1 |

[0154] +2×|P i-1，j-2 -P i-1，j |+2×|P i-1，j -P i-1，j+2 |+2×|P i+1，j-2 -P i+1，j |

[0155] +2×|P i+1，j -P i+1，j+2 |) / 32;

[0156] Calculate the same channel pixel difference value channel_diff_v in the vertical direction of the current target pixel, where the calculation formula is:

[0157] channel_diff_v

[0158] =(|P i-2，j-2 -P i，j-2 |+|P i，j-2 -P i+2，j-2 |+2×|P i-2，j -P i，j |+2

[0159] ×|P i，j -P i+2，j |+|P i-2，j+2 -P i，j+2 |+|P i，j+2 -P i+2，j+2 |+4

[0160] ×|P i-1，j-1 -P i+1，j-1 |+4×|P i-1，j+1 -P i+1，j+1 |+2

[0161] ×|P i-1，j-2 -P i+1，j-2 |+2×|P i-1，j+2 -P i+1，j+2 |+4×|P i-1，j -P i+1，j |

[0162] +2×|P i-2，j-1 -P i，j-1 |+2×|P i，j-1 -P i+2，j-1 |+2×|P i-2，j+1 -P i，j+1 |

[0163] +2×|P i，j+1 -P i+2，j+1 |) / 32;

[0164] Calculate the same channel pixel difference value channel_diff_d of the current target pixel in the diagonal direction, where the calculation formula is:

[0165] channel_diff_d

[0166] =(|P i-2，j -P i，j-2 |+|P i-2，j+2 -P i，j |+|P i+2，j-2 -P i，j |+|P i，j+2 -P i+2，j |

[0167] +2×(|P i-2，j+1 -P i，j-1 |+|P i-1，j -P i+1，j-2 |+|P i-1，j+2 -P i+1，j |

[0168] +|P i，j+1 -P i+2，j-1 |)+4×|Pi-1，j+1 -P i+1，j-1 |) / 16;

[0169] Calculate the same-channel pixel difference value channel_diff_ad of the current target pixel in the anti-diagonal direction, where the calculation formula is:

[0170] channel_diff_ad

[0171] =(|P i-2，j -P i，j+2 |+|P i-2，j-2 -P i，j |+|P i+2，j+2 -P i，j |+|P i，j-2 -P i+2，j |

[0172] +2×(|P i-2，j-1 -P i，j+1 |+|P i-1，j -P i+1，j+2 |+|P i-1，j-2 -P i+1，j |

[0173] +|P i，j-1 -P i+2，j+1 |)+4×|P i-1，j-1 -P i+1，j+1 |) / 16.

[0174] It can be found from the above calculation formula that the difference value calculation closer to the current target pixel has a higher weight, so the confidence of the same pixel difference value calculated when the cache window size is large can be guaranteed, thereby ensuring the accuracy of the subsequent interpolation direction selection and the accuracy of the interpolation calculation. In addition, this embodiment uses the difference of the same channel value to avoid the gradient difference caused by the channel color difference of the image itself, and effectively reduces the sawtooth effect.

[0175] by Figure 2 As an example, the 5×5 buffer window shown in the figure i，j That is the pixel value P of the current target pixel 2，2 , other pixels can be obtained according to the corresponding relationship, and then the difference values ​​of the same-channel pixels in all directions of the current target pixel in the 5×5 cache window can be obtained.

[0176] In other embodiments, if the size of the cache window is larger than 5×5, the above formula can be used to calculate the pixels within 5×5 with the current target pixel as the center. Of course, the calculation formula for the same-channel pixel difference value under cache windows of other sizes and the weights of each difference value can also be derived based on the above calculation formula, and this application will not go into details. Of course, in other embodiments, other weight values ​​can also be assigned to each difference value, but it must be satisfied that the difference value calculation closer to the current target pixel has a higher weight. In addition, considering the efficiency of image processing and the compatibility of hardware, the sum of the weights of the same-channel pixel difference value calculations in each direction should be 2 n , where n is a positive integer greater than 2.

[0177] It should be noted that the calculation of the same-channel pixel difference value after taking other weight values ​​without violating the main purpose of this application, as well as the calculation of the same-channel pixel difference value under other size cache windows should also fall within the scope of protection of this application.

[0178] S32, in the cache window, calculating the difference values ​​of adjacent pixels of the current target pixel in the horizontal direction, the vertical direction, the diagonal direction and the anti-diagonal direction.

[0179] Specifically, in this embodiment, a method for calculating the difference values ​​of adjacent pixels of the current target pixel in each direction is given:

[0180] Calculate the horizontal difference value color_diff_h of the adjacent pixels of the current target pixel, where the calculation formula is:

[0181] color_diff_h=(|P i-2，j-2 -P i-2，j-1 |+|P i-2，j-1 -P i-2，j |+|P i-2，j -P i-2，j+1 |

[0182] +|P i-2，j+1 -P i-2，j+2 |+|P i+2，j-2 -P i+2，j-1 |+|P i+2，j-1 -P i+2，j |

[0183] +|P i+2，j -P i+2，j+1 |+|P i+2，j+1 -P i+2，j+2 |+2

[0184] ×(|P i-1，j-2 -P i-1，j-1 |+|P i-1，j-1 -P i-1，j|+|P i-1，j -P i-1，j+1 |

[0185] +|P i-1，j+1 -P i-1，j+2 |+|P i，j-2 -P i，j-1 |+|P i，j-1 -P i，j |+|P i，j -P i，j+1 |

[0186] +|P i，j+1 -P i，j+2 |+|P i+1，j-2 -P i+1，j-1 |+|P i+1，j-1 -P i+1，j |

[0187] +|P i+1，j -P i+1，j+1 |+|P i+1，j+1 -P i+1，j+2 |)) / 32;

[0188] Calculate the vertical difference value color_diff_v of the adjacent pixels of the current target pixel, where the calculation formula is:

[0189] color_diff_v=(|P i-2，j-2 -P i-1，j-2 |+|P i-1，j-2 -P i，j-2 |+|P i，j-2 -P i+1，j-2 |

[0190] +|P i+1，j-2 -P i+2，j-2 |+|P i-2，j+2 -P i-1，j+2 |+|P i-1，j+2 -P i，j+2 |

[0191] +|P i，j+2 -P i+1，j+2 |+|P i+1，j+2 -P i+2，j+2 |+2

[0192] ×(|P i-2，j-1 -P i-1，j-1 |+|P i-1，j-1 -P i，j-1 |+|P i，j-1 -P i+1，j-1 |

[0193] +|P i+1，j-1 -Pi+2，j-1 |+|P i-2，j -P i-1，j |+|P i-1，j -P i，j |+|P i，j -P i+1，j |

[0194] +|P i+1，j -P i+2，j |+|P i-2，j+1 -P i-1，j+1 |+|P i-1，j+1 -P i，j+1 |

[0195] +|P i，j+1 -P i+1，j+1 |+|P i+1，j+1 -P i+2，j+1 |)) / 32;

[0196] Calculate the adjacent pixel difference value color_diff_d of the current target pixel in the diagonal direction, where the calculation formula is:

[0197] color_diff_d=(|P i-1，j-2 -P i-2，j-1 |+|P i，j-2 -P i-1，j-1 |+|P i-1，j-1 -P i-2，j |

[0198] +|P i+1，j-2 -P i，j-1 |+|P i，j-1 -P i-1，j |+|P i-1，j -P i-2，j+1 |

[0199] +|P i+2，j-2 -P i+1，j-1 |+|P i+1，j-1 -P i，j |+|P i，j -P i-1，j+1 |

[0200] +|P i-1，j+1 -P i-2，j+2 |+|P i+2，j-1 -P i+1，j |+|P i+1，j -P i，j+1 |

[0201] +|P i，j+1 -P i-1，j+2 |+|P i+2，j -P i+1，j+1 |+|Pi+1，j+1 -P i，j+2 |

[0202] +|P i+2，j+1 -P i+1，j+2 |) / 16;

[0203] Calculate the adjacent pixel difference value color_diff_ad of the current target pixel in the anti-diagonal direction, where the calculation formula is:

[0204] color_diff_ad

[0205] =(|P i-2，j+1 -P i-1，j+2 |+|P i-2，j -P i-1，j+1 |+|P i-1，j+1 -P i，j+2 |

[0206] +|P i-2，j-1 -P i-1，j |+|P i-1，j -P i，j+1 |+|P i，j+1 -P i+1，j+2 |

[0207] +|P i-2，j-2 -P i-1，j-1 |+|P i-1，j-1 -P i，j |+|P i，j -P i+1，j+1 |

[0208] +|P i+1，j+1 -P i+2，j+2 |+|P i-1，j-2 -P i，j-1 |+|P i，j-1 -P i+1，j |

[0209] +|P i+1，j -P i+2，j+1 |+|P i，j-2 -P i+1，j-1 |+|P i+1，j-1 -P i+2，j |

[0210] +|P i+1，j-2 -P i+2，j+1 |) / 16.

[0211] It can be found from the above calculation formula that the difference value calculation closer to the current target pixel has a higher weight, so the confidence of the difference value of adjacent pixels calculated when the cache window size is large can be guaranteed, thereby ensuring the accuracy of the selection of the subsequent interpolation direction and the accuracy of the interpolation calculation. In addition, this embodiment uses the difference of adjacent pixel points to improve the high-frequency recognition and restoration capabilities of the image, and effectively avoids the appearance of false colors in the image after de-mosaic interpolation.

[0212] by Figure 2 As an example, the 5×5 buffer window shown in the figure i,j That is the pixel value P of the current target pixel 2,2 , other pixels can be obtained according to the corresponding relationship, and then the difference values ​​of adjacent pixels in each direction of the current target pixel in the 5×5 cache window can be obtained.

[0213] In other embodiments, if the size of the cache window is larger than 5×5, the above formula can be used to calculate the pixels within 5×5 with the current target pixel as the center. Of course, the calculation formula for the difference values ​​of adjacent pixels under cache windows of other sizes and the weights of each difference value can also be derived based on the above calculation formula, and this application will not go into details. Of course, in other embodiments, other weight values ​​can also be assigned to each difference value, but it must be satisfied that the difference value calculation closer to the current target pixel has a higher weight. In addition, considering the efficiency of image processing and the compatibility of hardware, the sum of the weights of the adjacent pixel difference value calculations in each direction should be 2 n , where n is a positive integer greater than 2.

[0214] It should be noted that the calculation of adjacent pixel difference values ​​after taking other weight values ​​without violating the main purpose of this application, as well as the calculation of adjacent pixel difference values ​​under other size cache windows should also fall within the scope of protection of this application.

[0215] S33, calculating the color difference weight of the current target pixel in the cache window.

[0216] Specifically, in this embodiment, first, two adjacent color difference values ​​of the current target pixel in the horizontal direction and the vertical direction are calculated. Among them, the adjacent color difference value represents the color difference between two adjacent pixels, namely GR and GB. Therefore, in this embodiment, the two adjacent color difference values ​​in the horizontal direction are recorded as Cr_h (indicating the GR color difference value in the horizontal direction) and Cb_h (indicating the GB color difference value in the horizontal direction); similarly, the two adjacent color difference values ​​in the vertical direction are recorded as Cr_v and Cb_v. The specific calculation method of the adjacent color difference values ​​in the horizontal direction and the vertical direction can refer to the calculation method of the adjacent pixel difference values ​​of the current target pixel in the horizontal direction and the vertical direction mentioned above, and the calculation method well known to those skilled in the art can also be used, and this application does not limit this.

[0217] Then, the larger value of the two adjacent color difference values ​​in the horizontal direction is taken as the color difference value in the horizontal direction, and the larger value of the two adjacent color difference values ​​in the vertical direction is taken as the color difference value in the vertical direction. That is, the color difference value in the horizontal direction is: Color_bias_h = MAX(Cb_h, Cr_h), and the color difference value in the vertical direction is: Color_bias_v = MAX(Cb_v, Cr_v).

[0218] Next, the smaller value of the color difference value in the horizontal direction and the color difference value in the vertical direction is taken as the local color difference value. That is, the local color difference value is: color_bias=MIN(Color_bias_h, Color_bias_v).

[0219] Finally, according to the set weight range, the local color difference value is range mapped to obtain the color difference weight. The weight range includes the range surrounded by the minimum weight color_wt_low and the maximum weight color_wt_high. In practical applications, it is necessary to reasonably select the minimum weight color_wt_low and the maximum weight color_wt_high according to the actual situation.

[0220] In this embodiment, the horizontal direction and the vertical direction are selected for calculation when calculating the color difference value. However, in other embodiments, it can also be realized through color ratio (color ratio) or color difference (color difference) according to the statistical values ​​of different channels in the cache window. Among them, the calculation method of color ratio and color difference is well known to those skilled in the art, and this application will not repeat it. After obtaining the color ratio or color difference, it can still be mapped to the value range to obtain the weight. Those skilled in the art can implement this part of the process based on the above content in combination with the prior art, and this application will not repeat it.

[0221] S34, using the color difference weight, the same channel pixel difference value and the adjacent pixel difference value, the gradient value is calculated. That is, using the color difference weight, the same channel pixel difference value in the horizontal direction and the adjacent pixel difference value in the horizontal direction, the gradient value in the horizontal direction is calculated; using the color difference weight, the same channel pixel difference value in the vertical direction and the adjacent pixel difference value in the vertical direction, the gradient value in the vertical direction is calculated; using the color difference weight, the same channel pixel difference value in the diagonal direction and the adjacent pixel difference value in the diagonal direction, the gradient value in the diagonal direction is calculated; using the color difference weight, the same channel pixel difference value in the anti-diagonal direction and the adjacent pixel difference value in the anti-diagonal direction, the gradient value in the anti-diagonal direction is calculated.

[0222] Specifically, in this embodiment, the color difference weight is denoted as color_wt, where 0≤color_wt≤1, then:

[0223] The formula for calculating the gradient value grad_h in the horizontal direction can be expressed as:

[0224] grad_h=(1-color_wt)×channel_diff_h+color_wt×color_diff_h;

[0225] The formula for calculating the gradient value grad_v in the vertical direction can be expressed as:

[0226] grad_v=(1-color_wt)×channel_diff_v+color_wt×color_diff_v;

[0227] The formula for calculating the gradient value grad_d in the diagonal direction can be expressed as:

[0228] grad_d=(1-color_wt)×channel_diff_d+color_wt×color_diff_d;

[0229] The formula for calculating the gradient value grad_ad in the anti-diagonal direction can be expressed as:

[0230] grad_ad=(1-color_wt)×channel_diff_ad+color_wt×color_diff_ad.

[0231] This embodiment calculates the same-channel difference and the adjacent pixel (different channel) difference during the gradient value calculation process in four directions, and then obtains the color difference weight by mapping the screened local color difference value range. When calculating the gradient value, the weight is used to perform weighted summation on the same-channel pixel difference value and the adjacent pixel difference value, so that the calculation of the gradient value can adaptively obtain the participation weights of the two groups of differences, taking into account the black and white high-frequency area and the high color difference area.

[0232] Further, in this embodiment, in step S4, the method for determining the interpolation direction according to the gradient value in the horizontal direction, the gradient value in the vertical direction, the gradient value in the diagonal direction, and the gradient value in the anti-diagonal direction is as follows: Figure 4 As shown, including:

[0233] S41, setting the no-directional gradient threshold iso_th. The set no-directional gradient threshold iso_th is used to determine whether the current target pixel is in a flat area. Therefore, in practical applications, the value of the no-directional gradient threshold iso_th can be reasonably set according to actual conditions.

[0234] S42: If the horizontal gradient value grad_h, the vertical gradient value grad_v, the diagonal gradient value grad_d, and the anti-diagonal gradient value grad_ad of the current target pixel are all less than the non-directional gradient threshold iso_th, the current target pixel is determined to be a non-directional interpolation point.

[0235] S43: otherwise, determine the interpolation direction according to the gradient value grad_h in the horizontal direction, the gradient value grad_v in the vertical direction, the gradient value grad_d in the diagonal direction, and the gradient value grad_ad in the anti-diagonal direction.

[0236] Specifically, in this embodiment, the principle for selecting the main interpolation direction is that the gradient along this direction is relatively small, and the gradient along the direction perpendicular to this direction is relatively large. Therefore, two judgment conditions are set in judging the interpolation direction:

[0237] Judgment condition 1: Determine whether the ratio of the minimum value to the maximum value of the gradient value in the horizontal direction and the gradient value in the vertical direction is less than the ratio of the minimum value to the maximum value of the gradient value in the diagonal direction and the gradient value in the anti-diagonal direction, that is, whether MIN(grad_h,grad_v) / MAX(grad_h,grad_v)≤MIN(grad_d,grad_ad) / MAX(grad_d,grad_ad) holds;

[0238] Judgment condition 2: Determine whether the gradient value in the horizontal direction is less than the gradient value in the vertical direction, that is, whether grad_h ≤ grad_v holds.

[0239] Determine the interpolation direction based on the judgment result, specifically:

[0240] The gradient values ​​in the horizontal direction and the gradient values ​​in the vertical direction are taken as a group, and the gradient values ​​in the diagonal direction and the gradient values ​​in the anti-diagonal direction are taken as a group; the ratio of the smaller gradient value to the larger gradient value in each group is calculated, and the group with the smaller ratio in the two groups is taken as the direction candidate group; the direction corresponding to the smaller gradient value is selected from the direction candidate group as the interpolation direction.

[0241] This can be divided into the following situations:

[0242] A. If the ratio of the smaller value to the larger value of the gradient value in the horizontal direction and the gradient value in the vertical direction is less than or equal to the ratio of the smaller value to the larger value of the gradient value in the diagonal direction and the gradient value in the anti-diagonal direction, and the gradient value in the horizontal direction is less than or equal to the gradient value in the vertical direction, then the horizontal direction is selected as the interpolation direction; that is, MIN(grad_h,grad_v) / MAX(grad_h,grad_v)≤MIN(grad_d,grad_ad) / MAX(grad_d,grad_ad), and grad_h≤grad_v, then the horizontal direction is selected as the interpolation direction.

[0243] B. If the ratio of the smaller value to the larger value of the gradient value in the horizontal direction and the gradient value in the vertical direction is less than or equal to the ratio of the smaller value to the larger value of the gradient value in the diagonal direction and the gradient value in the anti-diagonal direction, and the gradient value in the horizontal direction is greater than the gradient value in the vertical direction, then the vertical direction is selected as the interpolation direction; that is, MIN(grad_h,grad_v) / MAX(grad_h,grad_v)≤MIN(grad_d,grad_ad) / MAX(grad_d,grad_ad), and grad_h>grad_v, then the vertical direction is selected as the interpolation direction.

[0244] C. If the ratio of the smaller value to the larger value of the gradient value in the horizontal direction and the gradient value in the vertical direction is greater than the ratio of the smaller value to the larger value of the gradient value in the diagonal direction and the gradient value in the anti-diagonal direction, and the gradient value in the diagonal direction is less than or equal to the gradient value in the anti-diagonal direction, then the diagonal direction is selected as the interpolation direction; that is, MIN(grad_h,grad_v) / MAX(grad_h,grad_v)>

[0245] MIN(grad_d,grad_ad) / MAX(grad_d,grad_ad), and grad_d≤grad_ad, then the diagonal direction is selected as the interpolation direction.

[0246] D. If the ratio of the smaller value to the larger value of the gradient value in the horizontal direction and the gradient value in the vertical direction is greater than the ratio of the smaller value to the larger value of the gradient value in the diagonal direction and the gradient value in the anti-diagonal direction, and the gradient value in the diagonal direction is greater than the gradient value in the anti-diagonal direction, then the anti-diagonal direction is selected as the interpolation direction; that is, MIN(grad_h,grad_v) / MAX(grad_h,grad_v)>

[0247] MIN(grad_d,grad_ad) / MAX(grad_d,grad_ad), and grad_d>grad_ad, then the anti-diagonal direction is selected as the interpolation direction.

[0248] Furthermore, in this embodiment, in step S5, the method of performing RGB full channel interpolation calculation along the interpolation direction to output a demosaiced image includes:

[0249] S51, extracting pixels of the same color on both sides of the current target pixel in the interpolation direction.

[0250] Specifically, in this embodiment, the pixel values ​​of two pixels of the same color are respectively P a,b and P c,d , where the values ​​of a and c are or The values ​​of b and d are or The specific values ​​of the same-color pixels need to be determined according to the interpolation direction, and the two same-color pixels are located on both sides of the interpolation direction of the current target pixel.

[0251] For example, in this embodiment, Figure 2 As shown, assuming that the selected interpolation direction is horizontal, then the pixel with the current target pixel (pixel value is P 2,2 ) The pixel values ​​of two adjacent pixels of the same color are recorded as P 2,0 and P 2,4 Of course, different adjacent pixels of the same color can be extracted according to different interpolation directions. At this time, G pixels (2,1) and (2,3) are interpolation candidate points, and their pixel values ​​P 2,1 and P 2,3 is the interpolation candidate pixel value.

[0252] S52, using the difference relationship between the two same-color pixels and the current target pixel, calculate the interpolation weights on both sides corresponding to the interpolation direction.

[0253] Specifically, in this embodiment, first, the same color difference values ​​on both sides of the interpolation direction of the current target pixel are calculated, where the calculation formula is:

[0254] Diff_1=|P i,j -P a,b |

[0255] Diff_2=|P i,j -P c,d |

[0256] Diff=|Diff_1-Diff_2|

[0257] Where Diff_1 is the difference value of the same color pixels on one side of the interpolation direction, Diff_2 is the difference value of the same color pixels on the other side of the interpolation direction, and Diff is the difference value of the same color on both sides of the interpolation direction of the current target pixel, that is, the difference calculated by comparing the difference values ​​on both sides;

[0258] Still Figure 2 As shown in the example, Diff_1 = |P 2,2 -P 2,0 |, Diff_2 = |P 2,2 -P 2,4 |.

[0259] Then, according to the set endpoint threshold value range, the same color difference values ​​on both sides of the interpolation direction of the current target pixel are range mapped to obtain the interpolation weight wt, where 0≤wt≤1.

[0260] S53, calculating the channel value of the to-be-interpolated color channel of the current target pixel according to the interpolation weight.

[0261] Specifically, in this embodiment, first, the color difference of the current target pixel on both sides corresponding to the interpolation direction is calculated, where the calculation formula is:

[0262]

[0263] Among them, Cr_1 is the color difference of one side of the current target pixel in the interpolation direction, Cr_2 is the color difference of the other side of the current target pixel in the interpolation direction, and Cr is the composite color difference of the current target pixel.

[0264] Still Figure 2 As shown in the example,

[0265] Next, the channel pixel value of the color channel to be interpolated of the current target pixel is calculated, and the calculation formula is:

[0266] Pixel C =P i,j -Cr.

[0267] For example, for Figure 2 As shown, the current target pixel (2,2) is an R pixel, and the G pixel value obtained by interpolation calculation can be expressed as: G_final = P 2,2 -Cr.

[0268] Similarly, the RGB channel values ​​of each pixel of the entire RAW image can be obtained, thereby converting the RAW image into an RGB image to obtain a demosaiced image.

[0269] In this embodiment, only pixels of the same color are used to obtain interpolation weights and synthetic color differences. Of course, in other embodiments, interpolation weights and synthetic color differences can also be obtained by combining differences of other channels. In addition, in order to reduce the influence of noise on the interpolation calculation results during interpolation calculation, filtering processing can also be performed before interpolation calculation.

[0270] Since in this embodiment, the pixels on both sides of the current target pixel in the interpolation direction are used in the interpolation calculation, the interpolation calculation in eight directions can be realized by judging four interpolation directions, thereby ensuring the clarity of the line endpoints and reducing pseudo-color.

[0271] The local color difference adaptive image de-mosaic interpolation method provided in this embodiment uses the first-order gradient of the difference between adjacent pixels to ensure the recognition and restoration of high-frequency information to the greatest extent; uses the local color difference information to guide the difference between adjacent pixels to participate in the calculation weight of the directional gradient, and has good adaptability to the area with color deviation, especially the color line area; by interpolating the directional color difference and calculating the color channel value to be interpolated with the color value of the current target pixel itself, the generation of interpolation pseudo-color is reduced. Therefore, the local color difference adaptive de-mosaic interpolation method provided in this embodiment can improve the image resolution, reduce the interpolation pseudo-color, and has better adaptability in images or areas where black and white high-frequency lines and color high-frequency lines exist.

[0272] The local color difference adaptive image de-mosaic interpolation method provided in this embodiment, when performing interpolation calculation, adaptively calculates the respective participation weights through the same channel differences in the same direction, expands the four main directions into actual eight-direction interpolation, improves the accuracy of the interpolation direction, and reduces the blur and pseudo color of the line endpoints.

[0273] Compared with the existing interpolation method, the local chromatic aberration adaptive image de-mosaicing interpolation method provided in this embodiment reduces the dependence of the 8-direction gradient calculation on the window size through the gradient calculation in 4 main directions, and reduces the problem of the gradual decrease in confidence of distant points; at the same time, while ensuring that the pixel values ​​in each direction are the same, the size of the cache window is halved, which significantly reduces the consumption of hardware resources.

[0274] This embodiment also provides a system-level chip, such as Figure 6 As shown, the system-level chip includes a processor and a memory, the processor and the memory are coupled, and the processor is used to execute a computer program stored in the memory to perform the local color difference adaptive image de-mosaic interpolation method as described above.

[0275] This embodiment further provides a computer storage medium, characterized in that the computer storage medium stores an executable program; when the executable program is executed, the above-mentioned local color difference adaptive image demosaicing interpolation method is implemented.

[0276] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other. In addition, the different parts between the various embodiments can also be used in combination with each other, and the present invention is not limited to this.

[0277] The local chromatic aberration adaptive image demosaicing interpolation method, system-level chip, and computer storage medium provided in this embodiment can adaptively obtain the weights of the pixel points participating in the interpolation according to the difference between the center point and the two ends of the main direction during the interpolation process, thereby reducing the size of the calculation window while ensuring the interpolation accuracy. Therefore, during the interpolation calculation, not only can the resolution of the image be effectively improved, the quality of the image after demosaicing is improved, but also the consumption of hardware resources can be saved, thereby solving the problem that the interpolation method used in the existing demosaicing process has poor accuracy and high hardware consumption, resulting in poor image quality.

[0278] The above description is only a description of the preferred embodiments of the present invention, and is not intended to limit the scope of the present invention. Any changes or modifications made by a person skilled in the art in the field of the present invention based on the above disclosure shall fall within the scope of protection of the claims.

Claims

1. A local color difference adaptive image demosaicing interpolation method, characterized in that: include: Get the original RAW image; Extract the cache window based on the current target pixel in the RAW image; In the cache window, calculate the gradient value of the current target pixel in the horizontal direction, vertical direction, diagonal direction and anti-diagonal direction; Determine the interpolation direction according to the gradient value in the horizontal direction, the gradient value in the vertical direction, the gradient value in the diagonal direction, and the gradient value in the anti-diagonal direction; Along the interpolation direction, RGB full channel interpolation calculation is performed to output the demosaiced image.

2. The local color difference adaptive image demosaicing interpolation method according to claim 1, characterized in that: The step of extracting the cache window based on the current target pixel in the RAW image comprises: Taking the current target pixel in the RAW image as a reference, extract a cache window of m rows and n columns, where m is an odd number greater than or equal to 5, and n is an odd number greater than or equal to 5; The pixel value of the pixel in the cache window is recorded as P x,y , where x represents the row coordinate of the pixel in the cache window, in the order from left to right, x = 0, 1, ..., m-1; y represents the column coordinate of the pixel in the cache window, in the order from top to bottom, y = 0, 1, ..., n-1; The pixel value of the current target pixel is recorded as P i,j ,in, 3. The local color difference adaptive image demosaicing interpolation method according to claim 2, characterized in that: The step of calculating the gradient value of the current target pixel in the horizontal direction, the vertical direction, the diagonal direction and the anti-diagonal direction in the cache window includes: In the cache window, calculate the same-channel pixel difference values ​​of the current target pixel in the horizontal direction, vertical direction, diagonal direction and anti-diagonal direction; In the cache window, calculate the difference values ​​of adjacent pixels of the current target pixel in the horizontal direction, vertical direction, diagonal direction and anti-diagonal direction; In the cache window, calculate the color difference weight of the current target pixel; The gradient value in the horizontal direction is calculated using the color difference weight, the difference value of the same-channel pixels in the horizontal direction, and the difference value of the adjacent pixels in the horizontal direction; the gradient value in the vertical direction is calculated using the color difference weight, the difference value of the same-channel pixels in the vertical direction, and the difference value of the adjacent pixels in the vertical direction; the gradient value in the diagonal direction is calculated using the color difference weight, the difference value of the same-channel pixels in the diagonal direction, and the difference value of the adjacent pixels in the diagonal direction; the gradient value in the anti-diagonal direction is calculated using the color difference weight, the difference value of the same-channel pixels in the anti-diagonal direction, and the difference value of the adjacent pixels in the anti-diagonal direction.

4. The local color difference adaptive image demosaicing interpolation method according to claim 3, characterized in that: The step of calculating the same-channel pixel difference values ​​of the current target pixel in the horizontal direction, the vertical direction, the diagonal direction and the anti-diagonal direction in the cache window includes: Calculate the pixel difference value channel_diff_h of the same channel of the current target pixel in the horizontal direction, where the calculation formula is: channel_diff_h =(|P i-2,j-2 -P i-2,j |+|P i-2,j -P i-2,j+2 |+2×|P i,j-2 -P i,j |+2×|P i,j -P i,j+2 |+|P i+2,j-2 -P i+2,j |+|P i+2,j -P i+2,j+2 |+4×|P i-1,j-1 -P i-1,j+1 |+4×|P i+1,j-1 -P i+1,j+1 |+2×|P i-2,j-1 -P i-2,j+1 |+2×|P i+2,j-1 -P i+2,j+1 |+4×|P i,j-1 -P i,j+1 |+2×|P i-1,j-2 -P i-1,j |+2×|P i-1,j -P i-1,j+2 |+2×|P i+1,j-2 -P i+1,j |+2×|P i+1,j -P i+1,j+2 |) / 32; Calculate the same channel pixel difference value channel_diff_v in the vertical direction of the current target pixel, where the calculation formula is: channel_diff_v =(|P i-2,j-2 -P i,j-2 |+|P i,j-2 -P i+2,j-2 |+2×|P i-2,j -P i,j |+2×|P i,j -P i+2,j |+|P i-2,j+2 -P i,j+2 |+|P i,j+2 -P i+2,j+2 |+4×|P i-1,j-1 -P i+1,j-1 |+4×|P i-1,j+1 -P i+1,j+1 |+2×|P i-1,j-2 -P i+1,j-2 |+2×|P i-1,j+2 -P i+1,j+2 |+4×|P i-1,j -P i+1,j |+2×|P i-2,j-1 -P i,j-1 |+2×|P i,j-1 -P i+2,j-1 |+2×|P i-2,j+1 -P i,j+1 |+2×|P i,j+1 -P i+2,j+1 |) / 32; Calculate the same channel pixel difference value channel_diff_d of the current target pixel in the diagonal direction, where the calculation formula is: channel_diff_d =(|P i-2,j -P i,j-2 |+|P i-2,j+2 -P i,j |+|P i+2,j-2 -P i,j |+|P i,j+2 -P i+2,j |+2×(|P i-2,j+1 -P i,j-1 |+|P i-1,j -P i+1,j-2 |+|P i-1,j+2 -P i+1,j |+|P i,j+1 -P i+2,j-1 |)+4×|P i-1,j+1 -P i+1,j-1 |) / 16; Calculate the same-channel pixel difference value channel_diff_ad of the current target pixel in the anti-diagonal direction, where the calculation formula is: channel_diff_ad =(|P i-2,j -P i,j+2 |+|P i-2,j-2 -P i,j |+|P i+2,j+2 -P i,j |+|P i,j-2 -P i+2,j |+2×(|P i-2,j-1 -P i,j+1 |+|P i-1,j -P i+1,j+2 |+|P i-1,j-2 -P i+1,j |+|P i,j-1 -P i+2,j+1 |)+4×|P i-1,j-1 -P i+1,j+1 |) / 16。 5. The local color difference adaptive image demosaicing interpolation method according to claim 3, characterized in that: The step of calculating the difference values ​​of adjacent pixels of the current target pixel in the horizontal direction, the vertical direction, the diagonal direction and the anti-diagonal direction in the cache window comprises: Calculate the horizontal difference value color_diff_h of the adjacent pixels of the current target pixel, where the calculation formula is: color_diff_h=(|P i-2,j-2 -P i-2,j-1 |+|P i-2,j-1 -P i-2,j |+|P i-2,j -P i-2,j+1 |+|P i-2,j+1 -P i-2,j+2 |+|P i+2,j-2 -P i+2,j-1 |+|P i+2,j-1 -P i+2,j |+|P i+2,j -P i+2,j+1 |+|P i+2,j+1 -P i+2,i+2 |+2×(|P i-1,j-2 -P i-1,j-1 |+|P i-1,j-1 -P i-1,j |+|P i-1,j -P i-1,j+1 |+|P i-1,j+1 -P i-1,j+2 |+|P i,j-2 -P i,j-1 |+|P i,j-1 -P i,j |+|P i,j -P i,j+1 |+|P i,j+1 -P i,j+2 |+|P i+1,j-2 -P i+1,j-1 |+|P i+1,j-1 -P i+1,j |+|P i+1,j -P i+1,j+1 |+|P i+1,j+1 -P i+1,j+2 |)) / 32; Calculate the vertical difference value color_diff_v of the adjacent pixels of the current target pixel, where the calculation formula is: color_diff_v=(|P i-2,j-2 -P i-1,j-2 |+|P i-1,j-2 -P i,j-2 |+|P i,j-2 -P i+1,j-2 |+|P i+1,j-2 -P i+2,j-2 |+|P i-2,j+2 -P i-1,j+2 |+|P i-1,j+2 -P i,j+2 |+|P i,j+2 -P i+1,j+2 |+|P i+1,j+2 -P i+2,j+2 |+2×(|P i-2,j-1 -P i-1,i-1 |+|P i-1,j-1 -P i,j-1 |+|P i,i-1 -P i+1,j-1 |+|P i+1,j-1 -P i+2,j-1 |+|P i-2,j -P i-1,j |+|P i-1,j -P i,j |+|P i,j -P i+1,j |+|P i+1,j -P i+2,j |+|P i-2,j+1 -P i-1,j+1 |+|P i-1,i+1 -P i,j+1 |+|P i,j+1 -P i+1,j+1 |+|P i+1,j+1 -P i+2,j+1 |)) / 32; Calculate the adjacent pixel difference value color_diff_d of the current target pixel in the diagonal direction, where the calculation formula is: color_diff_d=(|P i-1,j-2 -P i-2,j-1 |+|P i,j-2 -P i-1,j-1 |+|P i-1,j-1 -P i-2,j |+|P i+1,j-2 -P i,j-1 |+|P i,j-1 -P i-1,j |+|P i-1,j -P i-2,j+1 |+|P i+2,j-2 -P i+1,j-1 |+|P i+1,j-1 -P i,j |+|P i,j -P i-1,j+1 |+|P i-1,j+1 -P i-2,j+2 |+|P i+2,j-1 -P i+1,j |+|P i+1,j -P i,j+1 |+|P i,j+1 -P i-1,j+2 |+|P i+2,j -P i+1,j+1 |+|P i+1,j+1 -P i,j+2 |+|P i+2,j+1 -P i+1,j+2 |) / 16; Calculate the adjacent pixel difference value color_diff_ad of the current target pixel in the anti-diagonal direction, where the calculation formula is: color_diff_ad =(|P i-2,j+1 -P i-1,j+2 |+|P i-2,j -P i-1,j+1 |+|P i-1,j+1 -P i,j+2 |+|P i-2,j-1 -P i-1,j |+|P i-1,j -P i,j+1 |+|P i,j+1 -P i+1,j+2 |+|P i-2,i-2 -P i-1,j-1 |+|P i-1,j-1 -P i,j |+|P i,j -P i+1,j+1 |+|P i+1,j+1 -P i+2,j+2 |+|P i-1,j-2 -P i,j-1 |+|P i,j-1 -P i+1,j |+|P i+1,i -P i+2,j+1 |+|P i,j-2 -P i+1,j-1 |+|P i+1,j-1 -P i+2,j |+|P i+1,j-2 -P i+2,j+1 |) / 16。 6. The local color difference adaptive image demosaic interpolation method according to claim 3, characterized in that: The step of calculating the color difference weight of the current target pixel in the cache window includes: Calculate the two adjacent color difference values ​​of the current target pixel in the horizontal direction and the vertical direction; The larger value of the two adjacent color difference values ​​in the horizontal direction is taken as the color difference value in the horizontal direction, and the larger value of the two adjacent color difference values ​​in the vertical direction is taken as the color difference value in the vertical direction; The smaller value between the color difference value in the horizontal direction and the color difference value in the vertical direction is taken as the local color difference value; According to the set weight range, the local color difference value is range mapped to obtain the color difference weight.

7. The local color difference adaptive image demosaic interpolation method according to claim 3, characterized in that: The color difference weight is color_wt, 0≤color_wt≤1; The formula for calculating the gradient value grad_h in the horizontal direction is: grad_h=(1-color_wt)×channel_diff_h+color_wt×color_diff_h; The formula for calculating the gradient value grad_v in the vertical direction is: grad_v=(1-color_wt)×channel_diff_v+color_wt×color_diff_v; The formula for calculating the gradient value grad_d in the diagonal direction is: grad_d=(1-color_wt)×channel_diff_d+color_wt×color_diff_d; The formula for calculating the gradient value grad_ad in the anti-diagonal direction is: grad_ad=(1-color_wt)×channel_diff_ad+color_wt×color_diff_ad.

8. The local color difference adaptive image demosaicing interpolation method according to claim 1, characterized in that: The step of determining the interpolation direction according to the gradient value in the horizontal direction, the gradient value in the vertical direction, the gradient value in the diagonal direction, and the gradient value in the anti-diagonal direction comprises: Set the undirected gradient threshold; If the horizontal gradient value, the vertical gradient value, the diagonal gradient value and the anti-diagonal gradient value of the current target pixel are all less than the non-directional gradient threshold, the current target pixel is judged to be a non-directional interpolation point; Otherwise, the interpolation direction is determined according to the gradient value in the horizontal direction, the gradient value in the vertical direction, the gradient value in the diagonal direction, and the gradient value in the anti-diagonal direction.

9. The local color difference adaptive image demosaicing interpolation method according to claim 8, characterized in that: The step of determining the interpolation direction according to the gradient value in the horizontal direction, the gradient value in the vertical direction, the gradient value in the diagonal direction, and the gradient value in the anti-diagonal direction comprises: The gradient values ​​in the horizontal direction and the gradient values ​​in the vertical direction are grouped together, and the gradient values ​​in the diagonal direction and the gradient values ​​in the anti-diagonal direction are grouped together; Calculate the ratio of the smaller gradient value to the larger gradient value in each group, and take the group with the smaller ratio as the candidate direction group; The direction corresponding to the smaller gradient value is selected from the direction candidate group as the interpolation direction.

10. The local color difference adaptive image demosaic interpolation method according to claim 2, characterized in that: The step of performing RGB full channel interpolation calculation along the interpolation direction includes: In the interpolation direction, extract the same-color pixels on both sides of the current target pixel; The interpolation weights on both sides of the interpolation direction are calculated using the difference between two pixels of the same color and the current target pixel; The channel value of the color channel to be interpolated of the current target pixel is calculated according to the interpolation weight.

11. The local color difference adaptive image demosaicing interpolation method according to claim 10, characterized in that: The step of calculating the interpolation weights on both sides of the interpolation direction using the difference relationship between the two same-color pixels and the current target pixel includes: The pixel values ​​of two pixels of the same color are P a,b and P c,d , where the values ​​of a and c are or The values ​​of b and d are or Calculate the color difference on both sides of the interpolation direction of the current target pixel, where the calculation formula is: Diff_1=|P i,j -P a,b | Diff_2=|P i,j -P c,d | Diff=|Diff_1-Diff_2| Where Diff_1 is the difference value of the same color pixels on one side of the interpolation direction, Diff_2 is the difference value of the same color pixels on the other side of the interpolation direction, and Diff is the difference value of the same color pixels on both sides of the interpolation direction of the current target pixel; According to the set endpoint threshold value range, the same color difference values ​​on both sides of the interpolation direction of the current target pixel are range mapped to obtain the interpolation weight wt, where 0≤wt≤1.

12. The local color difference adaptive image demosaicing interpolation method according to claim 11, characterized in that: The step of calculating the channel value of the color channel to be interpolated of the current target pixel according to the interpolation weight comprises: Calculate the color difference of the current target pixel on both sides of the interpolation direction, where the calculation formula is: Wherein, Cr_1 is the color difference of one side of the current target pixel in the interpolation direction, Cr_2 is the color difference of the other side of the current target pixel in the interpolation direction, and Cr is the composite color difference of the current target pixel; Calculate the channel pixel value of the color channel to be interpolated of the current target pixel, where the calculation formula is: Pixel C =P i,j -Cr。 13. A system-on-chip, characterized in that: The system-level chip includes a processor and a memory, the processor and the memory are coupled, and the processor is used to execute a computer program stored in the memory to perform the local color difference adaptive image demosaicing interpolation method as described in any one of claims 1 to 12.

14. A computer storage medium, characterized in that: The computer storage medium stores an executable program; when the executable program is executed, the local color difference adaptive image demosaicing interpolation method according to any one of claims 1 to 12 is implemented.

Images