An image bad point processing method, an electronic device, and a storage medium

Abstract

A method for processing image bad points, comprising: acquiring a color filter matrix of 5*5 pixels centered on a pixel point to be tested; calculating gradients in four directions of horizontal, vertical, negative diagonal and positive diagonal respectively with the center as a reference, and finding a minimum gradient direction; selecting a plurality of reference pixel points from the minimum gradient direction and the center periphery, and sorting pixel values of the plurality of reference pixel points to form a reference pixel interval for bad point detection; when a pixel value of the pixel point to be tested exceeds a preset threshold outside the reference pixel interval, determining that it is a bad point; and replacing the bad point with a pixel value of a reference pixel point in the reference pixel interval. The bad point processing method provided by the application can effectively detect and correct multiple and continuous bad points in different regions of an image.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image defect processing method, electronic device, and storage medium. Background Technology

[0002] Despite significant advancements in image sensor technology, manufacturing processes for image sensors still suffer from shortcomings. Each image sensor inherently contains some defective pixels, or dead pixels. Furthermore, the number of dead pixels increases with prolonged use and exposure to high temperatures.

[0003] Traditional dead pixel correction methods first require image sensor manufacturers to calibrate the coordinates of dead pixels, then directly look up tables to find and correct them. However, this approach fails to correct new dead pixels that appear during use. In recent years, gradient-based dead pixel detection and correction methods have emerged, primarily targeting dead pixels at image edges. However, these methods are prone to missing high-frequency dead pixels, and existing solutions cannot effectively handle multiple dead pixels. The mutual interference between consecutive dead pixels also leads to poor dead pixel correction results. Summary of the Invention

[0004] This application provides an image defect processing method and an image sensor that can process image defects indiscriminately, and has a good processing effect on defects in different regions of the image (low frequency and high frequency regions).

[0005] In a first aspect, a method for processing bad pixels in an image is provided. The method includes: obtaining a 5*5 pixel color filtering matrix centered on the pixel to be tested; calculating gradients in four directions—horizontal, vertical, negative diagonal, and positive diagonal—based on the center, and finding the direction of minimum gradient; selecting multiple reference pixels from the direction of minimum gradient and the periphery of the center, and sorting the pixel values ​​of the multiple reference pixels to form a reference pixel interval for bad pixel detection; determining a bad pixel when the pixel value of the pixel to be tested exceeds a preset threshold outside the reference pixel interval; and replacing the bad pixel with the pixel value of one of the reference pixels in the reference pixel interval.

[0006] In one feasible implementation, the step of calculating gradients in the four directions (horizontal, vertical, negative diagonal, and positive diagonal) based on the center and finding the direction with the minimum gradient further includes: calculating multiple absolute gradient values ​​for the G, B, and R channels in each of the four directions; adding the multiple absolute gradient values ​​for the G, B, and R channels in each of the four directions to obtain the sum of the four absolute gradient values; comparing the sum of the four absolute gradient values ​​and selecting the direction with the minimum sum of the absolute gradient values ​​as the minimum gradient direction.

[0007] In one possible implementation, the multiple absolute gradient values ​​of the G channel, B channel, and R channel include: four absolute gradient values ​​of the G channel, two absolute gradient values ​​of the B channel, and two absolute gradient values ​​of the R channel.

[0008] One possible implementation also includes:

[0009] The multiple absolute gradient values ​​of the G channel, B channel, and R channel include: 4 absolute gradient values ​​of the G channel, 2 absolute gradient values ​​of the B channel, and 2 absolute gradient values ​​of the R channel.

[0010] Sort the absolute values ​​of the four gradients of the G channel and replace the maximum value with the second largest value;

[0011] Sort the absolute values ​​of the four gradients of the R channel and the B channel, and replace the maximum value with the second largest value.

[0012] In one possible implementation, selecting multiple reference pixels from the minimum gradient direction and the center periphery includes:

[0013] Using the center as a reference, select 8 pixels of the same color as the center along the periphery of the 5*5 color filtering matrix to form a reference matrix. Then, select two reference pixels p1 and p2 in the reference matrix along the minimum gradient direction, and calculate the average pixel value of reference pixel p1 and reference pixel p2 to obtain reference pixel p3. Select four peripheral pixels around the center and process them to obtain reference pixels p4-p7. Combine the 7 reference pixels to form a reference pixel interval for bad pixel detection.

[0014] In one possible implementation, the selection of the four surrounding pixels includes:

[0015] If the B pixel or R pixel is the center, then the four G pixels Ga-Gd located above, below, left, and right of the center point are selected as the surrounding pixels; if the G pixel is the center, then the four same-color pixels G9-G12 located on the negative and positive diagonals of the center point and close to the center are selected as the surrounding pixels.

[0016] In one possible implementation, the processing of the four surrounding pixels includes:

[0017] If pixel B or pixel R is the center, then the component Ba-Bd or Ra-Rd of the same color as the center in Ga-Gd is recovered using local color ratio, and the component Ba-Bd or Ra-Rd of the same color is used as reference pixel points p4-p7; if pixel G is the center, then the four pixels of the same color G9-G12 are directly used as reference pixel points p4-p7.

[0018] In one possible implementation, the process for the local color ratio includes:

[0019] If pixel B is the center, obtain all B and G pixels in the 5*5 pixel color filter matrix region of the center; calculate the weighted average values ​​ave_b and ave_g of the B and G pixels respectively; calculate the blue component Ba-Bd of the surrounding pixels Ga-Gd based on the weighted average values:

[0020] Bi=Gi*ave_b / ave_g(i=a,b,c,d);

[0021] If pixel R is the center, obtain all R and G pixels in the 5*5 pixel color filter matrix region of the center; calculate the weighted average values ​​ave_r and ave_g of the R and G pixels respectively; calculate the red component Ra-Rd of the surrounding pixels Ga-Gd based on the weighted average values:

[0022] Ri=Gi*ave_r / ave_g(i=a,b,c,d).

[0023] One possible implementation includes: multiplying the B, R, or G pixels obtained from the color filtering matrix by their corresponding weights to obtain the weighted average values ​​ave_b, ave_r, or ave_g.

[0024] In one possible implementation, it may further include: sorting the B, R, or G pixels obtained from the color filtering matrix and removing the maximum and minimum values, and using the remaining points multiplied by the corresponding weights to obtain the weighted average value ave_b, ave_r, or ave_g.

[0025] In one possible implementation, the plurality of reference pixels can be sorted and the maximum and minimum values ​​can be removed, with the second largest value as the upper boundary and the second smallest value as the lower boundary to form the reference pixel interval.

[0026] In one possible implementation, determining a pixel as a bad pixel when its pixel value exceeds a preset threshold outside the reference pixel range includes: determining a pixel as a bright spot when its pixel value exceeds the upper boundary of the reference pixel range—a first preset threshold THR0; and determining a pixel as a dark spot when its pixel value exceeds the lower boundary of the reference pixel range—a second preset threshold THR1.

[0027] One possible implementation also includes:

[0028] If it is determined to be a bright spot, replace the bright spot with the upper boundary value of the reference pixel range;

[0029] If a dark spot is determined, the dark spot is replaced with the lower boundary value of the reference pixel range.

[0030] In one possible implementation, replacing the bad pixel with the pixel value of a reference pixel in the reference pixel range includes replacing the bad pixel with the median value of the pixel values ​​in the reference pixel range.

[0031] In one possible implementation, the first preset threshold THR0 and the second preset threshold THR1 are set based on the minimum value of the sum of the absolute values ​​of the gradients.

[0032] The formulas for setting the first preset threshold THR0 and the second preset threshold THR1 are as follows:

[0033] THR0 = a * MinGrad + offset0;

[0034] THR1 = b * MinGrad + offset1;

[0035] Where MinGrad = MIN_GRAD / 4, MIN_GRAD is the minimum sum of absolute gradient values, a and b are coefficients, and offset0 and offset1 are offsets.

[0036] In a second aspect, an electronic device is provided, comprising: a processor; and a memory for storing computer program instructions; wherein, when the computer program instructions are loaded and run by the processor, the processor performs the method in any possible implementation of the first aspect described above.

[0037] Thirdly, a computer-readable storage medium is provided having computer program instructions stored thereon, which, when loaded and run by a processor, cause the processor to perform the method in any possible implementation of the first aspect described above.

[0038] The present invention employs the selection of multiple reference pixels from the minimum gradient direction and the periphery of the center during defect detection and correction, thereby making the detection and correction of defective pixels in the image more accurate. Attached Figure Description

[0039] Figure 1 This is a flowchart of the image defect processing method provided in the embodiments of this application.

[0040] Figure 2 This is a schematic diagram of a 5*5 pixel color filter matrix centered on RGB, provided in an embodiment of this application.

[0041] Figure 3 This is the calculation process for the minimum gradient direction provided in the embodiments of this application.

[0042] Figure 4 This is a schematic diagram of the RGB pixel reference matrix provided in an embodiment of this application.

[0043] Figure 5 This is the process for obtaining the reference pixel range provided in the embodiments of this application.

[0044] Figure 6 This is a flowchart illustrating the acquisition of four peripheral pixels provided in an embodiment of this application.

[0045] Figure 7 This is a flowchart of a partial color ratio provided in an embodiment of this application.

[0046] Figure 8 This is the weight value matrix provided in the embodiments of this application.

[0047] Figure 9 These are the peripheral pixels after the local color ratio provided in the embodiments of this application.

[0048] Figure 10 This is the defect detection process provided in the embodiments of this application. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0050] The application principles in the embodiments of this application will be further described below with reference to the accompanying drawings.

[0051] Bad pixels in an image are caused by manufacturing defects in the array of light-collecting points (pixels) on an image sensor, or by errors in the conversion of light signals into electrical signals. These defects result in incorrect pixel information and inaccurate pixel values ​​in the image. If bad pixels exist in an image, they will affect the values ​​of surrounding pixels during subsequent interpolation and filtering processes in Image Signal Processing (ISP). Therefore, bad pixels need to be corrected before interpolation and filtering. How to effectively and accurately identify bad pixels is the focus of Defect Pixel Correction (DPC) research.

[0052] Therefore, this application provides an image defect processing method that can effectively and accurately identify and correct multiple consecutive defective pixels in different regions, further simplifying calculations and improving processing speed.

[0053] like Figure 1 The diagram shown is a flowchart of an image defect processing method provided in an embodiment of this application. The image defect processing method mainly includes the following steps:

[0054] S101: Obtain a 5*5 pixel color filter matrix (Bayer array) centered on the pixel to be tested;

[0055] like Figure 2 The diagram shown is a schematic of a 5*5 pixel color filtering matrix provided in an embodiment of this application, wherein... Figure 2 The center pixel of (a) is pixel B. Figure 2 (b) has a center pixel of G pixels. Figure 2 (c) has a center pixel of R. As can be seen from the comparison of the figures, for the color filter matrix with a center pixel of R, we only need to swap the positions of R and B in the array with a center pixel of B in (a).

[0056] S102: Using the center as a reference, calculate the gradient in each of the four directions (H: horizontal direction, V: vertical direction, P: positive diagonal direction, N: negative diagonal direction) and find the direction with the minimum gradient.

[0057] like Figure 3 The diagram shows the calculation flow of the minimum gradient direction provided in the embodiments of this application. (Refer to...) Figure 3 Specifically, the method for calculating the minimum gradient direction includes the following steps:

[0058] S201: Calculate the absolute values ​​of multiple gradients for the G channel, B channel, and R channel in each of the four directions;

[0059] Specifically, when the center pixel is B, its color filtering matrix diagram is as follows: Figure 2 As shown in (a).

[0060] The horizontal gradient is calculated for each color channel (GBR) using the following formula:

[0061] g_diff_h[4]=[|G32-G34| / 2,(|G12-G14| / 2+|G52-G54| / 2) / 2,(|G21-G23| / 2+|G23-G25| / 2) / 2,(|G41-G43| / 2+|G43-G45| / 2) / 2];

[0062] b_diff_h[2]=[(|B11-B13| / 2+|B13-B15| / 2) / 2,(|B51-B53| / 2+|B53-B55| / 2) / 2];

[0063] r_diff_h[2]=[|R22-R24| / 2,|R42-R44| / 2];

[0064] like Figure 2 As shown in (a), the vertical gradient is calculated for each color channel, using the following formula:

[0065] g_diff_v[4]=[|G23-G43| / 2,(|G21-G41| / 2+|G25-G45| / 2) / 2,(|G12-G32| / 2+|G32-G52| / 2) / 2,(|G14-G34| / 2+|G34-G54| / 2) / 2];

[0066] b_diff_v[2]=[(|B11-B31| / 2+|B31-B51| / 2) / 2,(|B15-B35| / 2+|B35-B55| / 2) / 2];

[0067] r_diff_v[2]=[|R22-R42| / 2,|R24-R44| / 2];

[0068] like Figure 2 As shown in (a), the gradient calculation is performed on the negative diagonal direction for each color channel, and the formula is as follows:

[0069] g_diff_n[4]=[|G21-G32| / 2,|G43-G54| / 2,|G12-G34| / 2,|G23-45| / 2];

[0070] b_diff_n[2]=[|B31-B53| / 2,|B13-B35| / 2];

[0071] r_diff_n[2]=[|R22-R44| / 2,|R22-R44| / 2];

[0072] like Figure 2 As shown in (a), the gradient is calculated for each color channel along the diagonal direction, using the following formula:

[0073] g_diff_p[4]=[|G41-G23| / 2,|G52-G43| / 2,|G32-G14| / 2,|G34-G25| / 2];

[0074] b_diff_p[2]=[|B31-B13| / 2,|B53-B35| / 2];

[0075] r_diff_p[2]=[|R24-R42| / 2,|R24-R42| / 2];

[0076] It should be understood that in the above formula, the absolute values ​​of the gradients in the four directions of horizontal, vertical, positive diagonal, and negative diagonal include the gradients of 4 G channels, 2 R channels, and 2 B channels. Since the B and R pixels in the color filter matrix each account for 1 / 4 and the G pixels account for 1 / 2, the number of gradients distributed in each channel in the above calculation is to balance the sampling rate ratio of each channel, that is, G:B:R = 2:1:1.

[0077] When the center pixel is a G pixel, the absolute values ​​of the gradients in the four directions of horizontal, vertical, positive diagonal, and negative diagonal are calculated in sequence, including the gradients of 4 G channels, 2 R channels, and 2 B channels.

[0078] Its color filtering matrix diagram is as follows Figure 2 As shown in (b), the horizontal gradient is calculated for each color channel, using the following formula:

[0079] g_diff_h[4]=[|G22-G24| / 2,|G42-G44| / 2,(|G11-G13| / 2+|G13-G15| / 2) / 2,(|G51-G53| / 2+|G53-G55| / 2) / 2];

[0080] b_diff_h[2]=[|B32-B34| / 2,(|B12-B14| / 2+|B52-B54| / 2) / 2];

[0081] r_diff_h[2]=[(|R21-R23| / 2+|R23-R25| / 2) / 2,(|R41-R43| / 2+|R43-R45| / 2) / 2];

[0082] like Figure 2 As shown in (b), the vertical gradient is calculated for each color channel, using the following formula:

[0083] g_diff_v[4]=[|G22-G42| / 2,|G24-G44| / 2,(|G11-G31| / 2+|G31-G51| / 2) / 2,(|G15-G35| / 2+|G35-G55| / 2) / 2];

[0084] b_diff_v[2]=[(|B12-B32| / 2+|B32-B52| / 2) / 2,(|B14-B34| / 2+|B34-B54| / 2) / 2];

[0085] r_diff_v[2]=[|R23-R43| / 2,(|R21-R41| / 2+|R25-R45| / 2) / 2];

[0086] like Figure 2 As shown in (b), the gradient is calculated for each color channel along the negative diagonal direction, using the following formula:

[0087] g_diff_n[4]=[|G31-G42| / 2,|G42-G53| / 2,|G22-G44| / 2,|G13-G35| / 2];

[0088] b_diff_n[2]=[|B12-B34| / 2,|B32-B54| / 2];

[0089] r_diff_n[2]=[|R21-R43| / 2,|R23-R45| / 2];

[0090] like Figure 2 As shown in (b), the gradient is calculated for each color channel along the diagonal direction, using the following formula:

[0091] g_diff_p[4]=[|G31-G13| / 2,|G42-G24| / 2,|G53-G44| / 2,|G44-G35| / 2];

[0092] b_diff_p[2]=[|B32-B14| / 2,|B52-B34| / 2];

[0093] r_diff_p[2]=[|R41-R23| / 2,|R43-R25| / 2];

[0094] When the center pixel is R, the absolute values ​​of the gradients in the four directions of horizontal, vertical, positive diagonal, and negative diagonal are calculated in sequence, including the gradients of the four G channels, the gradients of the two R channels, and the gradients of the two B channels.

[0095] Its color filtering matrix diagram is as follows Figure 2 As shown in (c), the horizontal gradient is calculated for each color channel, using the following formula:

[0096] g_diff_h[4]=[|G32-G34| / 2,(|G12-G14| / 2+|G52-G54| / 2) / 2,(|G21-G23| / 2+|G23-G25| / 2) / 2,(|G41-G43| / 2+|G43-G45| / 2) / 2];

[0097] b_diff_h[2]=[(|R11-R13| / 2+|R13-R15| / 2) / 2,(|R51-R53| / 2+|R53-R55| / 2) / 2];

[0098] r_diff_h[2]=[|B22-B24| / 2,|B42-B44| / 2];

[0099] like Figure 2 As shown in (c), the vertical gradient is calculated for each color channel, using the following formula:

[0100] g_diff_v[4]=[|G23-G43| / 2,(|G21-G41| / 2+|G25-G45| / 2) / 2,(|G12-G32| / 2+|G32-G52| / 2) / 2,(|G14-G34| / 2+|G34-G54| / 2) / 2];

[0101] b_diff_v[2]=[(|R11-R31| / 2+|R31-R51| / 2) / 2,(|R15-R35| / 2+|R35-R55| / 2) / 2];

[0102] r_diff_v[2]=[|B22-B42| / 2,|B24-B44| / 2];

[0103] like Figure 2 As shown in (c), the gradient is calculated along the negative diagonal direction for each color channel, using the following formula:

[0104] g_diff_n[4]=[|G21-G32| / 2,|G43-G54| / 2,|G12-G34| / 2,|G23-45| / 2];

[0105] b_diff_n[2]=[|R31-R53| / 2,|R13-R35| / 2];

[0106] r_diff_n[2]=[|B22-B44| / 2,|B22-B44| / 2];

[0107] like Figure 2 As shown in (c), the gradient is calculated for each color channel along the diagonal direction, using the following formula:

[0108] g_diff_p[4]=[|G41-G23| / 2,|G52-G43| / 2,|G32-G14| / 2,|G34-G25| / 2];

[0109] b_diff_p[2]=[|R31-R13| / 2,|R53-R35| / 2];

[0110] r_diff_p[2]=[|B24-B42| / 2,|B24-B42| / 2];

[0111] In a preferred embodiment, to eliminate the influence of surrounding bad pixels on the absolute value of the gradient, the absolute values ​​of the gradient in different directions, including horizontal, vertical, positive diagonal, and negative diagonal, are preprocessed.

[0112] Specifically, the preprocessing includes: taking the horizontal gradient as an example, sorting the four gradients in g_diff_h and replacing the maximum value with the second largest value; sorting the four gradients in b_diff_h and r_diff_h and replacing the maximum value with the second largest value. The preprocessing for the other three directions is the same as the horizontal gradient processing described above.

[0113] In another preferred embodiment, to eliminate the influence of consecutive bad pixels on the absolute value of the gradient, two or more maximum and minimum values ​​can be removed during the above preprocessing.

[0114] It should be understood that, since the absolute values ​​of the two gradients in r_diff_n and r_diff_p on the positive and negative diagonals are equal when the center pixel is a B pixel or an R pixel, when the absolute value of the gradient in r_diff_n and r_diff_p is the largest, the second largest value should be used to replace the two gradients in r_diff_n and r_diff_p.

[0115] The above preprocessing methods can improve the detection and correction capabilities for multiple or consecutive bad pixels, ensure the effectiveness of correction for multiple bad pixels, prevent the correction of multiple bad pixels from affecting each other, and reduce the mutual influence between consecutive bad pixels.

[0116] S202: Sum the absolute values ​​of the gradients of the G channel, B channel, and R channel in each of the four directions to obtain the sum of the four absolute gradient values. The calculation formula is as follows:

[0117] Grad_H=sum(g_diff_h+b_diff_h+r_diff_h);

[0118] Grad_V=sum(g_diff_v+b_diff_v+r_diff_v);

[0119] Grad_N=sum(g_diff_n+b_diff_n+r_diff_n);

[0120] Grad_P=sum(g_diff_p+b_diff_p+r_diff_p);

[0121] The gradients of the horizontal, vertical, negative diagonal, and positive diagonal lines are represented as Grad_H, Grad_V, Grad_N, and Grad_P, respectively.

[0122] S203: Compare the sum of the absolute values ​​of the four gradients, and select the direction with the minimum sum of the absolute values ​​of the gradients as the minimum gradient direction.

[0123] Find the minimum value of the sum of the gradients in the four directions, MIN_GRAD = min(Grad_H, Grad_V, Grad_N, Grad_P), and then determine the minimum gradient direction dir = H / V / P / N.

[0124] S103: Select multiple reference pixels from the minimum gradient direction and the center periphery, and sort the pixel values ​​of the multiple reference pixels to form a reference pixel interval for bad pixel detection;

[0125] Specifically, the maximum value of the pixel values ​​of the plurality of reference pixels is used as the upper boundary of bad pixel detection, and the minimum value of the pixel values ​​is used as the lower boundary of bad pixel detection, thus forming a reference pixel range for bad pixel detection.

[0126] Because pixel values ​​change rapidly in high-frequency regions, the calculated gradient direction is unreliable, and correction along the direction of minimum gradient may lead to errors. In this invention, during defect detection and correction, not only are points along the direction of minimum gradient selected, but also points near the test point are chosen as reference pixels. These nearby points are closer to the test point's value, thus making the detection and correction of defective pixels in high-frequency regions more accurate.

[0127] As an optional implementation, the selection of reference pixels is based on a reference matrix. Specifically, the reference matrix is ​​selected by taking a 5x5 color filter matrix centered on the current pixel to be tested, and symmetrically selecting 8 pixels of the same color to form a pixel reference matrix, such as... Figure 4 The diagram shown is a reference matrix diagram of RGB pixels provided in an embodiment of this application, wherein... Figure 4 (a) is the B pixel reference matrix, and eight pixels of the same color, B1-B8, are selected from the center pixel B0; 4(b) is the G pixel reference matrix, and eight pixels of the same color, G1-G8, are selected from the center pixel G0; 4(c) is the R pixel reference matrix, and eight pixels of the same color, R1-R8, are selected from the center pixel R0.

[0128] like Figure 5 The diagram illustrates the process for obtaining the reference pixel range provided in this application embodiment, specifically including:

[0129] S501: Using the center as a reference, select 8 pixels of the same color as the center along the periphery of the 5*5 color filter matrix to form a reference matrix, and select two reference pixels p1 and p2 in the reference matrix along the minimum gradient direction, and calculate the average pixel value of the reference pixel p1 and the reference pixel p2 to obtain the reference pixel p3.

[0130] The reference pixel is selected from the direction of minimum gradient. Specifically, with Figure 4 Based on the RGB pixel reference matrix shown, the pixel reference matrix is ​​determined according to the center pixel of the pixel to be measured. Figure 4 One of these methods involves selecting two points, p1 and p2, from the eight same-color pixels B1-B8, G1-G8, or R1-R8 in the pixel reference matrix, along the determined minimum gradient direction dir = H / V / P / N, and calculating the average value of the reference pixel points p1 and p2 to obtain the reference pixel point p3.

[0131] Optionally, when the center pixel is pixel B, such as Figure 4 (a):

[0132] If the minimum gradient direction dir = H, then p1 = B1, p2 = B5, p3 = (B1 + B5) / 2;

[0133] If the minimum gradient direction dir = P, then p1 = B4, p2 = B8, p3 = (B4 + B8) / 2;

[0134] The same logic applies to other directions.

[0135] When the center pixel is G, such as Figure 4 (b):

[0136] If the minimum gradient direction dir = H, then p1 = G1, p2 = G5, p3 = (G1 + G5) / 2;

[0137] If the minimum gradient direction dir = P, then p1 = G4, p2 = G8, p3 = (G4 + G8) / 2;

[0138] The same logic applies to other directions.

[0139] When the center pixel is R, such as Figure 4 (c):

[0140] If the minimum gradient direction dir = H, then p1 = R1, p2 = R5, p3 = (R1 + R5) / 2;

[0141] If the minimum gradient direction dir = P, then p1 = R4, p2 = R8, p3 = (R4 + R8) / 2;

[0142] The same logic applies to other directions.

[0143] S502: Select four peripheral pixels around the center and process them to obtain reference pixel points p4-p7;

[0144] In practice, selecting peripheral pixels as reference pixels requires symmetrically selecting four peripheral pixels around the center pixel and processing them to obtain reference pixel points p4-p7, such as... Figure 6 The diagram shown is a flowchart illustrating the acquisition of four peripheral pixels according to an embodiment of this application.

[0145] according to Figure 4 It can be seen that the surrounding pixels of the center pixel B0 are Ga-Gd, the surrounding pixels of the center pixel G0 are G9-G12, and the surrounding pixels of the center pixel R0 are Ga-Gd.

[0146] Specifically, the four surrounding pixels are selected as follows:

[0147] If the center pixel is a B pixel or an R pixel, then select the four G pixels Ga-Gd located above, below, left, and right of the center pixel B0 or ​​R0 as the surrounding pixels.

[0148] If G is the center pixel, then select four pixels of the same color, G9-G12, that are close to the center on the negative and positive diagonals of the center, centered on G0, as the surrounding pixels.

[0149] The processing for the four surrounding pixels includes:

[0150] If the center pixel is a B pixel or an R pixel, then the same color component Ba-Bd or Ra-Rd of Ga-Gd is recovered using the local color ratio, and the same color component is used as the surrounding pixel, that is, Ba-Bd or Ra-Rd is used as the reference pixel point p4-p7.

[0151] If the center pixel is a G pixel, then the four pixels of the same color on the negative and positive diagonals are used directly as the surrounding pixels, i.e., as shown below. Figure 4 (b) shows the G-pixel reference matrix, which uses four points G9-G12 on the diagonal as reference pixel points p4-p7.

[0152] like Figure 7The diagram shown is a flowchart of a partial color ratio provided in an embodiment of this application. (Reference) Figure 7 Specifically, the local color ratio process includes (taking the center pixel B as an example; when the center pixel is R, the local color ratio process of Ra-Rd follows the same logic):

[0153] (1) According to Figure 2 (a) shows a schematic diagram of the color filtering matrix for the center pixel B. All B and G pixels within the 5x5 pixel color filtering matrix region at the center are obtained. The weighted average values ​​of the B and G pixels in this region, ave_b and ave_g, are calculated. The weight values ​​are determined based on the distance between each pixel and the center pixel; the closer the distance, the larger the weight value. Figure 8 The image shows a weight value matrix provided in an embodiment of this application.

[0154] ave_b=(B11*6+B13*8+B15*6+B31*8+B35*8+B51*6+B53*8+B55*6) / 56;

[0155] ave_g=(G12*7+G14*7+G21*7+G23*12+G25*7+G32*12+G34*12+G41*7+G43*12+G45*7+G52*7+G54*7) / 104;

[0156] In a preferred embodiment, to eliminate the influence of surrounding bad pixels and further improve the detection and correction capability of multiple consecutive bad pixels, (B11, B13, B15, B31, B35, B51, B53, B55) are sorted and the maximum and minimum values ​​are removed. The remaining points are multiplied by their corresponding weights and the average value is calculated to obtain ave_b; (G12, G14, G21, G23, G25, G32, G34, G41, G43, G45, G52, G54) are sorted and the maximum and minimum values ​​are removed. The remaining points are multiplied by their corresponding weights and the average value is calculated to obtain ave_g.

[0157] (2) Calculate the blue component Ba-Bd of the surrounding pixel Ga-Gd based on the weighted average value;

[0158] Specifically, taking Ga as an example, the blue component at position Ga is calculated as follows:

[0159] Ba = Ga * ave_b / ave_g;

[0160] Similarly, Bb = Gb * ave_b / ave_g; Bc = Gc * ave_b / ave_g; Bd = Gd * ave_b / ave_g.

[0161] This application uses a local color ratio method to process the surrounding pixels to form reference pixels when the B or R pixel is the center. Since Ga-Gd is closest to the center pixel B0 or ​​R0, the B or R component can be recovered by using a local color ratio method, which can provide a more accurate reference for bad pixel detection and correction.

[0162] Figure 9 The diagram shows the peripheral pixels after local color ratio recovery provided in the embodiments of this application. (a) is a schematic diagram of recovering the blue component Ba-Bd of Ga-Gd using local color ratio recovery when the center pixel is B; (b) is a schematic diagram of recovering the red component Ra-Rd of Ga-Gd using local color ratio recovery when the center pixel is R.

[0163] S503: Combine 7 reference pixels to form a reference pixel range for bad pixel detection.

[0164] In summary, if B is the center pixel, then the reference pixel for bad pixel detection REF_PIX[7] = [p1,p2,p3,Ba,Bb,Bc,Bd];

[0165] If G is the center pixel, then the reference pixel for bad pixel detection REF_PIX[7] = [p1,p2,p3,G9,G10,G11,G12];

[0166] If R is the center pixel, then the reference pixel for bad pixel detection REF_PIX[7] = [p1,p2,p3,Ra,Rb,Rc,Rd];

[0167] The seven points of REF_PIX are sorted, and the maximum value is used as the upper boundary and the minimum value as the lower boundary to form a reference pixel range for bad pixel detection.

[0168] In a preferred embodiment, to eliminate the influence of surrounding bad pixels and further improve the detection and correction capability of multiple consecutive bad pixels, the seven points of REF_PIX are sorted and the maximum and minimum pixel values ​​are removed. The second largest pixel value is used as the upper boundary and the second smallest pixel value is used as the lower boundary to form a reference pixel range for bad pixel detection.

[0169] S104: When the pixel value of the pixel to be tested exceeds a preset threshold outside the reference pixel range, it is determined to be a bad pixel;

[0170] Figure 10 The diagram illustrates the defect detection process provided in this application embodiment. Further, defective pixels include bright spots and dark spots; more specifically, the detection process for bright spots and dark spots includes:

[0171] If the pixel value of the pixel to be tested exceeds the upper boundary of the reference pixel range by a preset threshold THR0, it is determined to be a bright spot; if the pixel value of the pixel to be tested exceeds the lower boundary of the reference pixel range by a preset threshold THR1, it is determined to be a dark spot; otherwise, it is determined to be a normal spot.

[0172] The formulas for setting the preset thresholds THR0 and THR1 are as follows:

[0173] THR0 = a * MinGrad + offset0;

[0174] THR1 = b * MinGrad + offset1;

[0175] Where MinGrad = MIN_GRAD / 4, a and b are coefficients, and offset0 and offset1 are offsets. The choice of coefficients is related to the intensity of bad pixel detection; the larger the coefficient, the fewer bad pixels are detected.

[0176] The threshold is set using the minimum value of the sum of directional gradients, MIN_GRAD. In low-frequency regions, the gradient changes are small, resulting in a smaller minimum directional gradient sum, MIN_GRAD, and a correspondingly lower threshold. In high-frequency regions, the gradient changes are large, resulting in a larger minimum directional gradient sum, MIN_GRAD, and a correspondingly larger threshold. This allows the threshold to adapt to both low-frequency and high-frequency regions, enabling indiscriminate detection and correction of bad pixels across the entire image area, thus improving processing efficiency and speed.

[0177] S105: Select the pixel value of a reference pixel in the reference pixel range to replace the bad pixel.

[0178] refer to Figure 10 As shown, more specifically, if the point to be tested is determined to be a bright spot, the value of the upper boundary of the reference pixel range is used to replace the bright spot; if the point to be tested is determined to be a dark spot, the value of the lower boundary of the reference pixel range is used to replace the dark spot.

[0179] In a preferred embodiment, the median value of the pixel values ​​in the reference pixel range can also be selected to replace and correct bright spots and dark spots.

[0180] This invention employs multiple reference pixels selected from the minimum gradient direction and the surrounding area of ​​the center during defect detection and correction, resulting in more accurate defect detection and correction. Furthermore, this invention considers the influence of other surrounding defective pixels on the pixel under test throughout the entire defect detection and correction process, and performs corresponding preprocessing, improving the ability to detect and correct multiple or consecutive defective pixels. This invention uses a local color ratio method to process surrounding pixels when the B or R pixel is the center pixel; by recovering the B or R components of the four surrounding G pixels, it can provide more accurate reference pixels for defect detection and correction. This invention utilizes the minimum value of the directional gradient sum to set a threshold, allowing the preset threshold to change with variations in low-frequency and high-frequency regions of the image, achieving effective detection and correction of defective pixels across the entire image area.

[0181] This application embodiment can also provide an electronic device, which includes a processor and a memory. The memory is used to store computer program instructions, wherein when the computer program instructions are loaded and run by the processor, the processor executes the image bad pixel detection and correction method in any of the above embodiments.

[0182] This application embodiment can also provide a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the image defect detection and correction method of any of the above embodiments. The specific implementation process and beneficial effects of the computer-readable storage medium provided in this application embodiment are as described above and will not be repeated here.

[0183] It should be noted that the above-described method embodiments of this application can be applied to a processor or implemented by a processor. A processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above methods.

[0184] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0185] It should be understood that in the embodiments of this application, "B corresponding to A" means that B is associated with A, and B can be determined based on A. However, it should also be understood that determining B based on A does not mean that B is determined solely based on A; B can also be determined based on A and / or other information.

[0186] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0187] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0188] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0189] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0190] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0191] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0192] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0193] It should be understood that the specific examples in the embodiments of this application are only for the purpose of helping those skilled in the art to better understand the embodiments of this application, and are not intended to limit the scope of the embodiments of this application. Those skilled in the art can make various improvements and modifications based on the above embodiments, and all such improvements or modifications fall within the protection scope of this application.

[0194] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for processing bad pixels in an image, characterized in that, include: Obtain a 5x5 pixel color filtering matrix centered on the pixel to be tested; Using the center as a reference, calculate the gradient in the four directions of horizontal, vertical, negative diagonal, and positive diagonal, and find the direction of minimum gradient; Multiple reference pixels are selected from the minimum gradient direction and the periphery of the center, and the pixel values ​​of the multiple reference pixels are sorted to form a reference pixel interval for bad pixel detection, wherein the periphery of the center is the area where the pixels adjacent to the center are located. When the pixel value of the pixel to be tested exceeds a preset threshold outside the reference pixel range, it is determined to be a bad pixel; The pixel value of the bad pixel is replaced by the pixel value of a reference pixel in the reference pixel range.

2. The method according to claim 1, characterized in that, The step of calculating the gradient in the four directions (horizontal, vertical, negative diagonal, and positive diagonal) based on the center and finding the direction of minimum gradient also includes: Calculate the absolute values ​​of multiple gradients for the G, B, and R channels in each of the four directions; The absolute values ​​of multiple gradients in the G channel, B channel, and R channel of each of the four directions are added together to obtain the sum of the four absolute gradient values. Compare the sum of the four absolute gradient values, and select the direction with the minimum sum of the absolute gradient values ​​as the minimum gradient direction.

3. The method according to claim 2, characterized in that, The absolute values ​​of multiple gradients in the G, B, and R channels for each direction include: The absolute gradient values ​​of the four G channels, the two B channels, and the two R channels.

4. The method according to claim 2, characterized in that, The absolute values ​​of multiple gradients in the G, B, and R channels for each direction include: The absolute gradient values ​​of the 4 G channels, the absolute gradient values ​​of the 2 B channels, and the absolute gradient values ​​of the 2 R channels; Sort the absolute values ​​of the four gradients of the G channel and replace the maximum value with the second largest value; Sort the absolute values ​​of the four gradients of the R channel and the B channel, and replace the maximum value with the second largest value.

5. The method according to claim 1, characterized in that, The step of selecting multiple reference pixels from the minimum gradient direction and the center periphery includes: Using the center as a reference, select 8 pixels of the same color as the center along the periphery of the 5*5 color filter matrix to form a reference matrix. Then, select two reference pixels in the reference matrix along the minimum gradient direction, namely reference pixel p1 and reference pixel p2. Calculate the average pixel value of reference pixel p1 and reference pixel p2 to obtain reference pixel p3. Four peripheral pixels are selected around the center and processed to obtain reference pixel points p4, p5, p6 and p7; The seven reference pixels p1 to p7 are combined to form a reference pixel range for bad pixel detection.

6. The method according to claim 5, characterized in that, The selection of the four peripheral pixels includes: If the center is a B pixel or an R pixel, then the four G pixels Ga, Gb, Gc and Gd located above, below and to the left and right of the center point are selected as the surrounding pixels. If pixel G is the center, then the four pixels of the same color G9, G10, G11 and G12 adjacent to the center on the negative diagonal and positive diagonal of the center point are selected as the surrounding pixels.

7. The method according to claim 6, characterized in that, The processing of the four surrounding pixels includes: If the center is a B pixel or an R pixel, then the components Ba, Bb, Bc and Bd or Ra, Rb, Rc and Rd of the same color as the center in the surrounding pixels Ga, Gb, Gc and Gd are recovered using local color ratio, and the components Ba, Bb, Bc and Bd or Ra, Rb, Rc and Rd of the same color are used as the reference pixel points p4, p5, p6 and p7. If pixel G is the center, then the four pixels of the same color G9, G10, G11 and G12 are directly used as the reference pixel points p4, p5, p6 and p7.

8. The method according to claim 7, characterized in that, The process for the local color ratio includes: If the center is a B pixel, obtain all B pixels and G pixels in the 5*5 pixel color filtering matrix region of the center; Calculate the weighted average values ​​ave_b and ave_g for B pixels and G pixels respectively; Based on the weighted average values ​​ave_b and ave_g, the blue components Ba, Bb, Bc, and Bd of the surrounding pixels Ga, Gb, Gc, and Gd are calculated as follows: Bi=Gi * ave_b / ave_g (i=a,b,c,d); If the center is an R pixel, obtain all R pixels and G pixels in the 5*5 pixel color filter matrix region of the center; Calculate the weighted average values ​​ave_r and ave_g for R pixels and G pixels respectively; The red components Ra, Rb, Rc, and Rd of the surrounding pixels Ga, Gb, Gc, and Gd are calculated based on the weighted average as follows: Ri=Gi * ave_r / ave_g (i=a,b,c,d).

9. The method according to claim 8, characterized in that, include: The weighted average values ​​ave_b, ave_r, or ave_g are obtained by multiplying the pixel values ​​of B, R, or G pixels obtained from the color filtering matrix by the corresponding set weights.

10. The method according to claim 8, characterized in that, Also includes: The pixel values ​​of B, R, or G pixels obtained from the color filtering matrix are sorted and the maximum and minimum values ​​are removed. The remaining pixels are multiplied by their corresponding weights to obtain the weighted average values ​​ave_b, ave_r, or ave_g.

11. The method according to claim 1, characterized in that, Also includes: The pixel values ​​of the multiple reference pixels are sorted and the maximum and minimum values ​​are removed. The second largest value is used as the upper boundary and the second smallest value is used as the lower boundary to form the reference pixel interval.

12. The method according to any one of claims 1-11, characterized in that, The determination of a pixel as a bad pixel when its pixel value exceeds a preset threshold outside the reference pixel range includes: When the pixel value of the pixel to be tested exceeds the upper boundary of the reference pixel interval—a first preset threshold THR0—it is determined to be a bright spot. When the pixel value of the pixel to be tested exceeds the lower boundary of the reference pixel interval—a second preset threshold THR1—it is determined to be a dark spot.

13. The method according to claim 12, characterized in that, include: If it is determined to be a bright spot, replace the bright spot with the upper boundary value of the reference pixel range; If a dark spot is determined, the dark spot is replaced with the lower boundary value of the reference pixel range.

14. The method according to any one of claims 1-11, characterized in that, The step of selecting a reference pixel from the reference pixel range to replace the bad pixel includes replacing the bad pixel with the median value of the pixel values ​​in the reference pixel range.

15. The method according to claim 12, characterized in that, The first preset threshold THR0 and the second preset threshold THR1 are set based on the minimum value of the sum of the absolute values ​​of the gradients. The formulas for setting the first preset threshold THR0 and the second preset threshold THR1 are as follows: THR0 = a * MinGrad + offset0; THR1 = b * MinGrad + offset1; Where MinGrad = MIN_GRAD / 4, MIN_GRAD is the minimum sum of absolute gradient values, a and b are coefficients, and offset0 and offset1 are offsets.

16. An electronic device comprising: processor; as well as Memory is used to store computer program instructions; Wherein, when the computer program instructions are loaded and run by the processor, the processor performs the method as described in any one of claims 1 to 15.

17. A computer-readable storage medium having stored thereon computer program instructions, which, when loaded and executed by a processor, cause the processor to perform the method as described in any one of claims 1 to 15.

Images