A dynamic bad pixel correction method capable of reducing edge blur and lowering power consumption area

Abstract

The present invention provides a dynamic bad pixel correction method capable of reducing edge blur and reducing power consumption and area, comprising the following steps: setting the arrangement of the RGB pixel array, with the pixel to be corrected as the central pixel; calculating the maximum value, minimum value, second largest value, and second smallest value of the pixels in the neighborhood of the R pixel; comparing the central pixel with the maximum value and the minimum value; judging whether the pixel to be corrected is a cross bad pixel; calculating the neighborhood pixel fluctuation index according to different parameters; and correcting the historical pixels. Beneficial effects of the present invention: The present invention improves the correction accuracy of RGB pixels by adopting different correction methods for pixels of different color channels. By extracting the background bad pixel pattern of the pixel to be corrected and the edge fluctuation information of the picture, the weakening of the image edge and the information lost by the dynamic bad pixel correction are reduced. According to the structure of the data flow of a specific chip, a solution for cyclic reading and writing of Memory is designed, which reduces half of the IO interface and reduces the power consumption and area of ​​the chip.

Description

Technical Field

[0001] The invention belongs to the technical field of pixel bad pixel correction, and in particular relates to a dynamic bad pixel correction method capable of reducing edge blur and lowering power consumption area. Background Art

[0002] In today's society, people's requirements for camera effects are increasing day by day, which is not only reflected in the pursuit of image clarity, but also includes comprehensive consideration of image details, color reproduction and dynamic range. However, due to the limitations of pixel manufacturing technology, pixel defects, that is, bad pixels, are always inevitable. Specifically, some pixels are significantly different from the surrounding pixels, such as white dots in a completely black environment, black dots in a bright environment, or the pixel state changes with changes in conditions such as exposure or temperature. These bad pixels will seriously affect the image quality, resulting in problems such as loss of details, reduced contrast and color distortion. Therefore, bad pixel correction is one of the key links to improve image quality.

[0003] Existing bad pixel correction schemes are divided into static bad pixel correction and dynamic bad pixel correction. Static bad pixel correction relies on a pre-generated bad pixel table, which records the coordinate information of all known bad pixels. In practical applications, the bad pixel is determined by comparing the coordinates of the current pixel with the coordinates in the bad pixel table, and then corrected by methods such as median filtering. The advantage of this method is that it can accurately correct known static bad pixels, but the disadvantage is that it cannot adapt to the later defects caused by long-term use, and it is also powerless for those bad pixels related to temperature or exposure conditions. The current mainstream dynamic bad pixel correction scheme is the neighborhood comparison maximum value method. The difference between the central pixel and its surrounding pixels is calculated to determine whether it is a bad pixel. If the value of the central pixel is much larger or smaller than the value of its neighboring pixels, it is determined to be a bad pixel. For pixels that have been identified as bad pixels, the maximum value of other pixels in its neighborhood is used to replace the bad pixel value, thereby avoiding the negative impact of bad pixels on image quality. However, this maximum value-based scheme is prone to incorrect correction due to factors such as noise, and it also blurs the edges of the image. In addition, since historical pixel information is required, the traditional solution needs to store the historical data of the previous four rows of pixels, read it out when new pixel data arrives, and write the corrected data in. For a single-clock controlled read and write memory, twice the number of IO interfaces is required for read and write ping-pong operations, which greatly increases the power consumption and area of ​​the chip. Summary of the invention

[0004] In view of this, the present invention aims to provide a dynamic bad pixel correction method capable of reducing edge blur and lowering power consumption area, so as to solve at least one problem existing in the above-mentioned prior art.

[0005] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0006] A dynamic bad pixel correction method capable of reducing edge blur and power consumption area comprises the following steps:

[0007] S1. Set the arrangement of the RGB pixel array, take the pixel to be corrected as the center pixel, and select the information of the neighboring pixels to complete the determination and correction of the bad pixel;

[0008] S2, calculate the maximum value, minimum value, second largest value, and second smallest value of the pixels in the neighborhood of pixel R;

[0009] S3, comparing the central pixel with the maximum value and the minimum value;

[0010] S4, determining whether the pixel to be corrected is a cross bad pixel;

[0011] S5, calculating the neighborhood pixel fluctuation index according to different parameters;

[0012] S6. Correct historical pixels when new data arrives.

[0013] Furthermore, in step S1, the arrangement of the RGB pixel array is set, with the pixel to be corrected as the center, and the information of the neighboring pixels is selected to complete the determination and correction of the bad pixels, including:

[0014] The arrangement of the RGB pixel array is in Bayer format, with the pixel to be corrected as the center pixel, and 7 columns × 5 rows of neighboring pixels around the center pixel are set;

[0015] When the pixel to be corrected is R, extract the information of the eight R pixels around the pixel to be corrected, and record the maximum value max1, the second maximum value max2, the minimum value min1, and the second minimum value min2;

[0016] The number of bits of each R pixel supports 10 bits / 12 bits, and only the first 8 bits of each R pixel are included in the calculation.

[0017] Furthermore, in step S2, the maximum value, minimum value, second largest value, and second smallest value of the pixels in the neighborhood of the R pixel are calculated, including:

[0018] Compare the 8 R pixels in pairs, and store the comparison results in eight 7-bit arrays a, b, c, d, e, f, g, and h respectively;

[0019] Set the pixel to be corrected to P(2,3), extract the information of the 8 R pixels around P(2,3), that is, the pixel values ​​of P(0,1), P(0,3), P(0,5), P(2,1), P(2,5), P(4,1), P(4,3), and P(4,5). Take the comparison of P(0,1) as an example. If P(0,1)>P(0,3), then a(0)=1, b(0)=1, otherwise a(0)=0, b(0)=1;

[0020] If P(0,1)>P(0,5), then a(1)=1, c(0)=0… and so on, and the eight arrays are filled up by comparing them one by one;

[0021] Then use eight 3-bit arrays A, B, C, D, E, F, G, H for further comparison, and record A = a(0) + a(1) + a(2) + a(3) + a(4) + a(5) + a(6), B = b(0) + b(1) + b(3) + b(4) + b(5) + b(6);

[0022] And so on, we get the values ​​of the eight arrays A, B, C, D, E, F, G, and H, and record them as MAX1(0)

[0023] =A(2)&A(1)&A(0), MAX2(0)=A(2)&A(1)&! A(0),MIN1(0)=! A(2)

[0024] &! A(1)&! A(0),MIN2(0)=! A(2)&! A(1)&A(0),MAX1(1)=B(2)&B(1)&B(0)…

[0025] The results of the pairwise comparisons are converted into four 8-bit one-hot codes: MAX1, MAX2, MIN1, and MIN2. The high bit position of the one-hot code is the position of the corresponding maximum value, minimum value, second largest value, and second smallest value.

[0026] Furthermore, in step S3, the central pixel is compared with the maximum value and the minimum value, including:

[0027] Compare the central pixel with the maximum and minimum values. If the pixel value of the central pixel is greater than the sum of the neighborhood maximum value and the white point threshold, the pixel is marked as a white bad pixel.

[0028] If the pixel value of the central pixel is less than the difference between the minimum value of the neighborhood and the black point threshold, the pixel is marked as a black bad pixel;

[0029] If none of them are satisfied, the pixel is marked as a normal point and no correction is performed;

[0030] When the pixel to be corrected is determined to be a white bad pixel, if the double bad pixel correction is enabled at this time, the maximum value of the neighboring pixels is compared to see whether it is greater than the second largest value plus the white threshold;

[0031] If the condition is satisfied, it is further determined whether the pixel is surrounded by cross-shaped bad pixels and the calculated neighborhood pixel fluctuation index is less than the preset value, then the pixel to be corrected is corrected to the second largest value, otherwise it is corrected to the maximum value;

[0032] If the maximum value of the neighborhood pixel is not greater than the second largest value plus the white threshold, then determine whether the pixel is surrounded by cross-shaped bad pixels and the calculated neighborhood pixel fluctuation index is less than the preset value. If it is satisfied, it is corrected to the second largest value. If not, it is corrected to the maximum value.

[0033] The same applies to the correction of black pixels.

[0034] Furthermore, in step S4, determining whether the pixel to be corrected is a cross bad pixel includes:

[0035] When judging whether it is a cross bad pixel, if the pixel to be corrected is a white bad pixel, then judge whether the pixel to be corrected is greater than the central white threshold, and at the same time, the four pixels above, below, left and right of the pixel are greater than the background white threshold. If all of them are satisfied, it is recorded as a cross bad pixel;

[0036] If the pixel to be corrected is a black bad pixel, it is determined whether the pixel to be corrected is smaller than the central black threshold and the four surrounding pixels are smaller than the background black threshold. If all of them are satisfied, it is also recorded as a cross bad pixel.

[0037] Furthermore, in step S5, the neighborhood pixel fluctuation index is calculated according to different parameters, including:

[0038] When performing white pixel correction, if the central pixel is an R / B pixel, take the average of P(0,2), P(0,4), P(1,1), P(1,5), P(3,1), P(3,5), P(4,2), and P(4,4), and record it as Mean_G. Take the average of P(1,0), P(1,6), P(3,0), and P(3,6), and record it as Mean_RB. Let threshold = diff × RB_eff, where diff is MAX1-MAX2 or P(2,3)-MAX2 according to the input, and RB_eff is the proportional coefficient.

[0039] Count the eight pixels around the pixel to be tested, namely P(1,2), P(1,3), P(1,4), P(2,2), P(2,4), P(3,1), P(3,2), P(3,4), the number of which exceeds the corresponding color average brightness + threshold, which is the neighborhood pixel fluctuation index;

[0040] If the central pixel is a G pixel, the average values ​​of P(0,2), P(0,4), P(2,0), P(6,0), P(4,2), P(4,4) and P(1,1), P(1,5), P(3,1), P(3,5) are calculated respectively, and recorded as Mean_R and Mean_G, and threshold is set as diff×G_eff, where diff is MAX1-MAX2 or P(2,3)-MAX2 according to the input, and G_eff is the proportional coefficient;

[0041] Count the eight pixels around the pixel to be tested, namely P(1,2), P(1,3), P(1,4), P(2,2), P(2,4), P(3,1), P(3,2), P(3,4), the number of which exceeds the corresponding color average brightness + threshold, which is the neighborhood pixel fluctuation index;

[0042] When correcting black pixels, let threshold = diff × G_eff, where diff is MIN2-MIN1 or MIN2-P(2,3) according to the input. Count the 8 pixels around the pixel to be tested, namely P(1,2), P(1,3), P(1,4), P(2,2), P(2,4), P(3,1), P(3,2), and P(3,4), whose values ​​are lower than the corresponding color average brightness - threshold, which is the neighborhood pixel fluctuation index.

[0043] Furthermore, in step S6, when new data arrives, the historical pixels are corrected, including:

[0044] Each clock will read in 4 pixels of data. In order to correct the historical pixels when new data arrives, the pixel data of the previous 4 rows needs to be stored;

[0045] If the pixel data is up to 12 bits, use 5 memory blocks to read and write respectively, and the 5 memory blocks are recorded as mem1, mem2, mem3, mem4, mem5 respectively;

[0046] When the first row of data arrives, read out the data of mem1, mem2, mem3, mem4 corresponding to the 4 columns, and write the newly arrived 4 pixel values ​​into mem5;

[0047] When the second row of pixel data arrives, read out the data of mem2, mem3, mem4, mem5 corresponding to the four columns, and write the new data into mem1;

[0048] When the third row of data arrives, read out the data of mem3, mem4, mem5, mem1 corresponding to the four columns, and write the newly arrived four pixel values ​​into mem2;

[0049] When the fourth row of pixel data arrives, read out the data of mem4, mem5, mem1, mem2 corresponding to the four columns, and write the new data into mem3;

[0050] When the fifth row of data arrives, read out the data of mem5, mem1, mem2, and mem3 corresponding to the four columns, and write the newly arrived four pixel values ​​into mem4;

[0051] When the sixth row of pixel data arrives, the data of mem1, mem2, mem3, and mem4 corresponding to the four columns are read out as in the first row, and the new data is written into mem5;

[0052] And so on, all data are read in and out.

[0053] An electronic device comprises a processor and a memory which is connected to the processor in communication and used to store instructions executable by the processor. The processor is used to execute the dynamic bad pixel correction method which can reduce edge blur and power consumption area.

[0054] A server includes at least one processor and a memory connected to the processor in communication, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor so that the at least one processor executes the dynamic bad pixel correction method capable of reducing edge blur and lowering power consumption area.

[0055] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the dynamic bad pixel correction method capable of reducing edge blur and lowering power consumption area is implemented.

[0056] Compared with the prior art, the dynamic bad pixel correction method of the present invention which can reduce edge blur and power consumption area has the following advantages:

[0057] The present invention discloses a dynamic bad pixel correction method capable of reducing edge blur and power consumption and area. The present invention improves the correction accuracy of RGB pixels by adopting different correction methods for pixels of different color channels. By extracting the background bad pixel pattern of the pixel to be corrected and the edge fluctuation information of the picture, the weakening of the image edge and the information of image loss caused by dynamic bad pixel correction are reduced. According to the structure of the data flow of a specific chip, a solution of cyclic reading and writing of memory is designed, which reduces half of the IO interface and reduces the power consumption and area of ​​the chip. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0059] Figure 1 Schematic diagram of an arrangement of pixels in the neighborhood of an R / B pixel according to an embodiment of the present invention;

[0060] Figure 2 Schematic diagram of a G pixel neighborhood pixel arrangement method according to an embodiment of the present invention;

[0061] Figure 3 A schematic diagram of a pixel input timing sequence according to an embodiment of the present invention;

[0062] Figure 4 This is a schematic diagram of the Memory call timing according to an embodiment of the present invention;

[0063] Figure 5 The figure is a flow chart of a dynamic bad pixel correction method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0065] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and the like are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", and the like may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0066] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood by specific circumstances.

[0067] The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0068] like Figures 1 to 5 As shown, a dynamic bad pixel correction method capable of reducing edge blur and lowering power consumption area includes the following steps:

[0069] S1. Set the arrangement of the RGB pixel array, take the pixel to be corrected as the center pixel, and select the information of the neighboring pixels to complete the determination and correction of the bad pixel;

[0070] S2, calculate the maximum value, minimum value, second largest value, and second smallest value of the pixels in the neighborhood of pixel R;

[0071] S3, comparing the central pixel with the maximum value and the minimum value;

[0072] S4, determining whether the pixel to be corrected is a cross bad pixel;

[0073] S5, calculating the neighborhood pixel fluctuation index according to different parameters;

[0074] S6. Correct historical pixels when new data arrives.

[0075] In a preferred embodiment of the present invention, the arrangement of the RGB pixel array is a Bayer format, such as Figure 1 As shown in the figure. With the pixel to be corrected as the center, the information of the surrounding 7 columns × 5 rows of neighboring pixels is needed to complete the determination and correction of bad pixels. When the pixel to be corrected is R, Figure 1 For example, the pixel to be corrected is P(2,3). Extract the information of the 8 R pixels around P(2,3), namely P(0,1), P(0,3), P(0,5), P(2,1), P(2,5), P(4,1), P(4,3), P(4,5), and record the maximum value max1, the second maximum value max2, the minimum value min1, and the second minimum value min2. The number of bits of the pixel supports 10bit / 12bit. In order to facilitate unification and reduce the chip area, this solution only includes the first 8bit of each pixel in the calculation.

[0076] In a preferred embodiment of the present invention, in the process of obtaining the maximum value of the neighboring pixels, the eight pixels are compared in pairs, and the comparison results are respectively stored in eight 7-bit arrays a, b, c, d, e, f, g, and h. Taking the comparison of P(0,1) as an example, if P(0,1)>P(0,3), then a(0)=1, b(0)=1, otherwise a(0)=0, b(0)=1. If P(0,1)>P(0,5), then a(1)=1, c(0)=0... and the eight arrays are filled up by comparison in sequence. Then use eight 3-bit arrays A, B, C, D, E, F, G, H for further comparison. Let A = a(0) + a(1) + a(2) + a(3) + a(4) + a(5) + a(6), B = b(0) + b(1) + b(3) + b(4) + b(5) + b(6), and so on to obtain the values ​​of the eight arrays A, B, C, D, E, F, G, H. Let MAX1(0) = A(2) & A(1) & A(0), MAX2(0) = A(2) & A(1) & ! A(0), MIN1(0) = ! A(2) & ! A(1) & ! A(0), MIN2(0) = ! A(2) & ! A(1)&A(0), MAX1(1)=B(2)&B(1)&B(0)…Convert the results of the pairwise comparisons into four 8-bit one-hot codes: MAX1, MAX2, MIN1, MIN2. The high-order position of the one-hot code is the position of the corresponding maximum value, minimum value, second largest value, and second smallest value.

[0077] In a preferred embodiment of the present invention, after obtaining the maximum value, minimum value, second largest value and second smallest value of the neighborhood pixels of the R pixel, the central pixel is compared with the maximum value and the minimum value. If the pixel value of the central pixel is greater than the sum of the neighborhood maximum value and the white point threshold, the pixel is marked as a white bad pixel; if the pixel value of the central pixel is less than the difference between the neighborhood minimum value and the black point threshold, the pixel is marked as a black bad pixel; if both are not met, the pixel is marked as a normal point and no correction is performed. When the pixel to be corrected is determined to be a white bad pixel, if the double bad pixel correction is enabled at this time, the maximum value of the neighboring pixels is compared to see if it is greater than the second largest value plus the white threshold. If it is satisfied, it is further determined whether the pixel is not surrounded by a cross-shaped bad pixel, and the neighborhood pixel fluctuation index calculated by (max1-max2) is less than the preset value, then the pixel to be corrected is corrected to the second largest value, otherwise it is corrected to the maximum value; if it is not satisfied that the maximum value of the neighboring pixels is greater than the second largest value plus the white threshold, it is determined whether the pixel is not surrounded by a cross-shaped bad pixel, and the neighborhood pixel fluctuation index calculated by (P(2,3)-max2) is less than the preset value, if it is satisfied, it is corrected to the second largest value, if not, it is corrected to the maximum value. The correction of black bad pixels is similar.

[0078] In a preferred embodiment of the present invention, when judging whether it is a cross bad pixel, if the pixel to be corrected is a white bad pixel, it is judged whether the pixel to be corrected is greater than the central white threshold, and the four pixels above, below, left and right of the pixel are greater than the background white threshold. If all of them are satisfied, it is recorded as a cross bad pixel. If the pixel to be corrected is a black bad pixel, it is judged whether the pixel to be corrected is less than the central black threshold, and the four surrounding pixels are less than the background black threshold. If all of them are satisfied, it is also recorded as a cross bad pixel.

[0079] In a preferred embodiment of the present invention, in order to better process the image edge, it is necessary to calculate the neighborhood pixel fluctuation index according to different parameters. When performing white pixel correction, if the central pixel is an R / B pixel, such as Figure 1 As shown, the average of P(0,2), P(0,4), P(1,1), P(1,5), P(3,1), P(3,5), P(4,2), P(4,4) is taken, denoted as Mean_G, the average of P(1,0), P(1,6), P(3,0), P(3,6) is taken, denoted as Mean_RB, and threshold = diff × RB_eff, where diff is MAX1-MAX2 or P(2,3)-MAX2 according to the input, and RB_eff is the proportional coefficient. The number of pixels that exceed the corresponding color average brightness + threshold in the 8 pixels around the pixel to be tested, namely P(1,2), P(1,3), P(1,4), P(2,2), P(2,4), P(3,1), P(3,2), P(3,4), is the neighborhood pixel fluctuation index. If the central pixel is a G pixel, Figure 2As shown, the average values ​​of P(0,2), P(0,4), P(2,0), P(6,0), P(4,2), P(4,4) and P(1,1), P(1,5), P(3,1), P(3,5) are calculated respectively, denoted as Mean_R and Mean_G, and threshold is set as diff×G_eff, where diff is MAX1-MAX2 or P(2,3)-MAX2 according to the input, and G_eff is the proportional coefficient. The number of pixels exceeding the corresponding color average brightness + threshold in the 8 pixels around the pixel to be tested, namely P(1,2), P(1,3), P(1,4), P(2,2), P(2,4), P(3,1), P(3,2), P(3,4), is the neighborhood pixel fluctuation index. When correcting black pixels, let threshold = diff × G_eff, where diff is MIN2-MIN1 or MIN2-P(2,3) according to the input. Count the 8 pixels around the pixel to be tested, namely P(1,2), P(1,3), P(1,4), P(2,2), P(2,4), P(3,1), P(3,2), and P(3,4), whose values ​​are lower than the corresponding color average brightness - threshold, which is the neighborhood pixel fluctuation index.

[0080] In a preferred embodiment of the present invention, each clock will read the data of 4 pixels in the row direction, and its timing diagram is as follows: Figure 3 As shown. In order to correct the historical pixels when new data arrives, the pixel data of the previous 4 rows needs to be stored. In order to realize the single-clock read and write ping-pong operation, 12×8=96-bit IO interfaces are required, totaling 4 memories. In order to reduce the chip area and power consumption occupied by the memory, this design adopts a design of reading and writing 5 memories separately, avoiding ping-pong operations, reducing half of the IO interfaces, and reducing the power consumption and area of ​​the chip. Figure 4As shown, the five blocks of memory are recorded as mem1, mem2, mem3, mem4, and mem5. When the first row of data arrives, the data of mem1, mem2, mem3, and mem4 corresponding to the four columns are read out, and the four newly arrived pixel values ​​are written into mem5. When the second row of pixel data arrives, the data of mem2, mem3, mem4, and mem5 corresponding to the four columns are read out, and the new data is written into mem1. When the third row of data arrives, the data of mem3, mem4, mem5, and mem1 corresponding to the four columns are read out, and the four newly arrived pixel values ​​are written into mem2. When the fourth row of pixel data arrives, the data of mem4, mem5, mem1, and mem2 corresponding to the four columns are read out, and the new data is written into mem3. When the fifth row of data arrives, read out the data of mem5, mem1, mem2, mem3 corresponding to the four columns, and write the newly arrived four pixel values ​​into mem4. When the sixth row of pixel data arrives, the same as the first row, read out the data of mem1, mem2, mem3, mem4 corresponding to the four columns, and write the new data into mem5. And so on, complete the reading in and out of all data.

[0081] The present invention also proposes an electronic device, comprising a processor and a memory connected to the processor for storing executable instructions of the processor, wherein the processor is used to execute the dynamic bad pixel correction method capable of reducing edge blur and lowering power consumption area.

[0082] The present invention also proposes a server, comprising at least one processor and a memory connected to the processor in communication, wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the processor so that the at least one processor executes the dynamic bad pixel correction method that can reduce edge blur and reduce power consumption area.

[0083] The present invention also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the dynamic bad pixel correction method capable of reducing edge blur and lowering power consumption area is implemented.

[0084] The present invention proposes a dynamic bad pixel correction scheme that can be used for an RGB pixel array. By comparing the pixel to be corrected with the neighboring pixels of the same color channel and designing different comparison schemes for different color channels, the accuracy of correction is increased. In the comparison process of the neighboring pixels, not only the maximum and minimum values ​​in the neighborhood are recorded, but also the form of the background bad pixels and the edge information are judged, thereby reducing the weakening of the image edge and the loss of image information. In the reading and writing process of the memory, the reading and writing of the pixel data are separated by recycling the memory, thereby reducing the power consumption and area of ​​the chip. The present invention is an effective bad pixel correction scheme.

[0085] Example 1

[0086] The pixel array of the CMOS image sensor chip is 1024×1280, and the pixel width is 12 bits. Figure 3 As shown, four pixels enter the data stream for each clock, and these four pixels need to be processed at the same time. The chip uses 5 memories with a bit width of 48 and a depth of 336, which are recorded as Mem1, Mem2, Mem3, Mem4, and Mem5. When the first row of pixel data arrives, the data is written to Mem5, and the corresponding column data is read from Mem1, Mem2, Mem3, and Mem4. The data in Mem3 is corrected for bad pixels, and the result is output. When the second row of pixel data arrives, the data is written to Mem1, and the corresponding column data is read from Mem2, Mem3, Mem4, and Mem5. The data in Mem4 is corrected for bad pixels, and the result is output. When the third row of pixel data arrives, the data is written to Mem2, and the corresponding column data is read from Mem3, Mem4, Mem5, and Mem1. The data in Mem5 is corrected for bad pixels, and the result is output. When the second row of pixel data arrives, the data is written into Mem3, and the corresponding column data is read out from Mem4, Mem5, Mem1, and Mem2, the data in Mem1 is corrected for bad pixels, and the result is output. When the first row of pixel data arrives, the data is written into Mem4, and the corresponding column data is read out from Mem5, Mem1, Mem2, and Mem3, the data in Mem3 is corrected for bad pixels, and the result is output. Every five rows, the Memory calling method returns to the first row, completing the Memory calling loop.

[0087] In the process of correcting data, different neighboring pixels are used for correction according to the different RGB channels of the corrected pixel. When the pixel to be corrected is R / B, the neighboring pixels are as follows: Figure 1 , the pixel to be corrected is P(2,3). Extract the information of the 8 R / B pixels around P(2,3), namely P(0,3), P(1,2), P(2,1), P(3,2), P(4,3), P(3,4), P(2,5), P(1,4), and record the maximum value max1, the second maximum value max2, the minimum value min1, and the second minimum value min2. If the pixel to be corrected is a G pixel, the neighboring pixels are as follows: Figure 2 , extract the maximum value, minimum value, second largest value, and second smallest value among P(0,1), P(0,3), P(0,5), P(2,1), P(2,5), P(4,1), P(4,3), and P(4,5).

[0088] In the process of comparing the maximum value, only the first 8 bits of the pixel are compared. The 8 pixels are compared in pairs, and the comparison results are respectively put into eight 7-bit arrays a, b, c, d, e, f, g, h. If P(0,1)>P(0,3), then a(0)=1, b(0)=1, otherwise a(0)=0, b(0)=1. If P(0,1)>P(0,5), then a(1)=1, c(0)=0... Compare in sequence to fill up the eight arrays. Then use eight 3-bit arrays A, B, C, D, E, F, G, H for further comparison. Let A = a(0) + a(1) + a(2) + a(3) + a(4) + a(5) + a(6), B = b(0) + b(1) + b(3) + b(4) + b(5) + b(6), and so on to obtain the values ​​of the eight arrays A, B, C, D, E, F, G, H. Let MAX1(0) =

[0089] A(2)&A(1)&A(0), MAX2(0)=A(2)&A(1)&! A(0),MIN1(0)=! A(2)

[0090] &!A(1)&!A(0),MIN2(0)=!A(2)&!A(1)&A(0),MAX1(1)=B(2)&B(1)&B(0)…Convert the results of the pairwise comparisons into four 8-bit one-hot codes: MAX1, MAX2, MIN1, MIN2. The high bit position of the one-hot code is the corresponding maximum value, minimum value, second largest value, and second smallest value.

[0091] If the central pixel value exceeds the sum of the neighborhood maximum value and the white point threshold, the central pixel is marked as a white bad pixel; if the central pixel value is lower than the difference between the neighborhood minimum value and the black point threshold, the central pixel is marked as a black bad pixel; if neither of them is satisfied, it is regarded as a normal pixel and no correction is required. Set the white threshold and the black threshold to 0x0f. For pixels determined as white bad pixels, if the double bad pixel correction function is enabled, it will be compared whether the neighborhood maximum value is greater than the second largest value plus the white threshold. If the condition is met, it is also necessary to check whether there are no cross-shaped bad pixels around the pixel, and use (MAX1-MAX2) as a parameter to calculate whether the neighborhood pixel fluctuation index is less than the preset value. If it is satisfied, the pixel to be corrected is corrected to the second largest value, otherwise it is corrected to the maximum value; if the condition that the neighborhood maximum value is greater than the second largest value plus the white threshold is not met, it is checked whether there are no cross-shaped bad pixels around, and (P(2,3)-MAX2) is used as a parameter to calculate whether the neighborhood pixel fluctuation index is less than the preset value. If it is satisfied, it is corrected to the second largest value, otherwise it is corrected to the maximum value. The preset value of the neighborhood pixel fluctuation index is 2. The correction process for black bad pixels is the same as that for white bad pixels.

[0092] When judging whether it is a cross-shaped bad pixel, if the pixel to be corrected is a white bad pixel, check whether the pixel is greater than the central white threshold, and whether the four pixels above, below, left and right are greater than the background white threshold, and take the central white threshold = background white threshold = 0xf0. If all are satisfied, it is marked as a cross-shaped bad pixel. If the pixel to be corrected is a black bad pixel, check whether the pixel is less than the central black threshold, and whether the four surrounding pixels are less than the background black threshold, and take the central black threshold = background black threshold = 0x0f. If all are satisfied, it is also marked as a cross-shaped bad pixel.

[0093] In order to optimize the processing of image edges, it is necessary to calculate the neighborhood pixel fluctuation index based on the parameters. When correcting white bad pixels, if the central pixel is an R / B pixel, according to Figure 1 As shown, the average value of the pixels at a specific position is taken, recorded as Mean_G and Mean_RB, and threshold is calculated as diff×RB_eff, where diff can be MAX1-MAX2 or P(2,3)-MAX2 according to different inputs, and RB_eff is the proportional coefficient, which is 0.25. The number of the 8 pixels around the pixel to be tested that exceed the average brightness of the corresponding color + threshold is counted, which is the neighborhood pixel fluctuation index. If the central pixel is a G pixel, such as Figure 2 As shown, the average values ​​of pixels at specific positions are calculated respectively, recorded as Mean_R and Mean_G, and threshold = diff × G_eff is calculated, where diff can be MAX1-MAX2 or P(2,3)-MAX2 according to different inputs, and G_eff is a proportional coefficient, which is taken as 0.25. The number of the 8 pixels around the pixel to be tested that exceed the average brightness of the corresponding color + threshold is counted, which is the neighborhood pixel fluctuation index. When correcting black bad pixels, threshold = diff × G_eff is calculated, where diff can be MIN2-MIN1 or MIN2-P(2,3) according to different inputs, and the number of the 8 pixels around the pixel to be tested that are lower than the average brightness of the corresponding color - threshold is counted, which is the neighborhood pixel fluctuation index.

[0094] The present invention aims at the problem that the traditional bad pixel correction scheme weakens the edge of the image and uses a large amount of memory to occupy the power consumption and area of ​​the chip. A dynamic bad pixel correction scheme that can be used for RGB pixel array is proposed. By comparing the pixel to be corrected with the neighboring pixels of the same color channel and designing different comparison schemes for different color channels, the accuracy of correction is increased; in the comparison process of the neighboring pixels, not only the maximum and minimum values ​​in the neighborhood are recorded, but also the judgment of the background bad pixel form and edge information is made, which reduces the weakening of the image edge and the loss of image information caused by dynamic bad pixel correction; in the reading and writing process of memory, the pixel data is read and written in different memories at the same time by recycling the memory, which reduces half of the IO interface and reduces the power consumption and area of ​​the chip. The key point protected by the present invention is the algorithm of dynamic bad pixel correction and the corresponding digital chip design method.

[0095] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A dynamic bad pixel correction method capable of reducing edge blur and lowering power consumption area, characterized in that: The following steps are involved: S1. Set the arrangement of the RGB pixel array, take the pixel to be corrected as the center pixel, and select the information of the neighboring pixels to complete the determination and correction of the bad pixel; S2, calculate the maximum value, minimum value, second largest value, and second smallest value of the pixels in the neighborhood of pixel R; S3, comparing the central pixel with the maximum value and the minimum value; S4, determining whether the pixel to be corrected is a cross bad pixel; S5, calculating the neighborhood pixel fluctuation index according to different parameters; S6. Correct historical pixels when new data arrives.

2. A dynamic bad pixel correction method capable of reducing edge blur and power consumption area according to claim 1, characterized in that: In step S1, the arrangement of the RGB pixel array is set, with the pixel to be corrected as the center, and the information of the neighboring pixels is selected to complete the determination and correction of the bad pixels, including: The arrangement of the RGB pixel array is in Bayer format, with the pixel to be corrected as the center pixel, and 7 columns × 5 rows of neighboring pixels around the center pixel are set; When the pixel to be corrected is R, extract the information of the eight R pixels around the pixel to be corrected, and record the maximum value max1, the second maximum value max2, the minimum value min1, and the second minimum value min2; The number of bits of each R pixel supports 10 bits / 12 bits, and only the first 8 bits of each R pixel are included in the calculation.

3. The method for dynamic bad pixel correction capable of reducing edge blur and power consumption area according to claim 2, characterized in that: In step S2, the maximum value, minimum value, second maximum value, and second minimum value of the pixels in the neighborhood of the R pixel are calculated, including: Compare the 8 R pixels in pairs, and store the comparison results in eight 7-bit arrays a, b, c, d, e, f, g, and h respectively; Set the pixel to be corrected to P(2,3), extract the information of the 8 R pixels around P(2,3), that is, the pixel values ​​of P(0,1), P(0,3), P(0,5), P(2,1), P(2,5), P(4,1), P(4,3), and P(4,5). Take the comparison of P(0,1) as an example. If P(0,1)>P(0,3), then a(0)=1, b(0)=1, otherwise a(0)=0, b(0)=1; If P(0,1)>P(0,5), then a(1)=1, c(0)=0… and so on, and the eight arrays are filled up by comparing them one by one; Then use eight 3-bit arrays A, B, C, D, E, F, G, H for further comparison, and record A = a(0) + a(1) + a(2) + a(3) + a(4) + a(5) + a(6), B = b(0) + b(1) + b(3) + b(4) + b(5) + b(6); And so on, we can get the values ​​of the eight arrays A, B, C, D, E, F, G, and H. Let MAX1(0) = A(2) & A(1) & A(0), MAX2(0) = A(2) & A(1) & !A(0), MIN1(0) = !A(2) & !A(1) & !A(0), MIN2(0) = !A(2) & !A(1) & A(0), MAX1(1) = B(2) & B(1) & B(0)… The results of the pairwise comparisons are converted into four 8-bit one-hot codes: MAX1, MAX2, MIN1, and MIN2. The high bit position of the one-hot code is the position of the corresponding maximum value, minimum value, second largest value, and second smallest value.

4. The method for dynamic bad pixel correction capable of reducing edge blur and power consumption area according to claim 1, characterized in that: In step S3, the central pixel is compared with the maximum value and the minimum value, including: Compare the central pixel with the maximum and minimum values. If the pixel value of the central pixel is greater than the sum of the neighborhood maximum value and the white point threshold, the pixel is marked as a white bad pixel. If the pixel value of the central pixel is less than the difference between the minimum value of the neighborhood and the black point threshold, the pixel is marked as a black bad pixel; If none of them are satisfied, the pixel is marked as a normal point and no correction is performed; When the pixel to be corrected is determined to be a white bad pixel, if the double bad pixel correction is enabled at this time, the maximum value of the neighboring pixels is compared to see whether it is greater than the second largest value plus the white threshold; If the condition is satisfied, it is further determined whether the pixel is surrounded by cross-shaped bad pixels and the calculated neighborhood pixel fluctuation index is less than the preset value, then the pixel to be corrected is corrected to the second largest value, otherwise it is corrected to the maximum value; If the maximum value of the neighborhood pixel is not greater than the second largest value plus the white threshold, then determine whether the pixel is surrounded by cross-shaped bad pixels and the calculated neighborhood pixel fluctuation index is less than the preset value. If it is satisfied, it is corrected to the second largest value. If not, it is corrected to the maximum value. The same applies to the correction of black pixels.

5. The method for dynamic bad pixel correction capable of reducing edge blur and power consumption area according to claim 1, characterized in that: In step S4, determining whether the pixel to be corrected is a cross bad pixel includes: When judging whether it is a cross bad pixel, if the pixel to be corrected is a white bad pixel, then judge whether the pixel to be corrected is greater than the central white threshold, and at the same time, the four pixels above, below, left and right of the pixel are greater than the background white threshold. If all of them are satisfied, it is recorded as a cross bad pixel; If the pixel to be corrected is a black bad pixel, it is determined whether the pixel to be corrected is smaller than the central black threshold and the four surrounding pixels are smaller than the background black threshold. If all of them are satisfied, it is also recorded as a cross bad pixel.

6. The method for dynamic bad pixel correction capable of reducing edge blur and power consumption area according to claim 1, characterized in that: In step S5, the neighborhood pixel fluctuation index is calculated according to different parameters, including: When performing white pixel correction, if the central pixel is an R / B pixel, take the average of P(0,2), P(0,4), P(1,1), P(1,5), P(3,1), P(3,5), P(4,2), and P(4,4), and record it as Mean_G. Take the average of P(1,0), P(1,6), P(3,0), and P(3,6), and record it as Mean_RB. Let threshold = diff × RB_eff, where diff is MAX1-MAX2 or P(2,3)-MAX2 according to the input, and RB_eff is the proportional coefficient. Count the eight pixels around the pixel to be tested, namely P(1,2), P(1,3), P(1,4), P(2,2), P(2,4), P(3,1), P(3,2), P(3,4), the number of which exceeds the corresponding color average brightness + threshold, which is the neighborhood pixel fluctuation index; If the central pixel is a G pixel, the average values ​​of P(0,2), P(0,4), P(2,0), P(6,0), P(4,2), P(4,4) and P(1,1), P(1,5), P(3,1), P(3,5) are calculated respectively, and recorded as Mean_R and Mean_G, and threshold is set as diff×G_eff, where diff is MAX1-MAX2 or P(2,3)-MAX2 according to the input, and G_eff is the proportional coefficient; Count the eight pixels around the pixel to be tested, namely P(1,2), P(1,3), P(1,4), P(2,2), P(2,4), P(3,1), P(3,2), P(3,4), the number of which exceeds the corresponding color average brightness + threshold, which is the neighborhood pixel fluctuation index; When correcting black pixels, let threshold = diff × G_eff, where diff is MIN2-MIN1 or MIN2-P(2,3) according to the input. Count the 8 pixels around the pixel to be tested, namely P(1,2), P(1,3), P(1,4), P(2,2), P(2,4), P(3,1), P(3,2), and P(3,4), whose values ​​are lower than the corresponding color average brightness - threshold, which is the neighborhood pixel fluctuation index.

7. The method for dynamic bad pixel correction capable of reducing edge blur and power consumption area according to claim 1, characterized in that: In step S6, when new data arrives, historical pixels are corrected, including: Each clock will read in the data of 4 pixels in the row direction. In order to correct the historical pixels when new data arrives, the pixel data of the previous 4 rows needs to be stored; Use 5 memory blocks for reading and writing respectively, and the 5 memory blocks are respectively recorded as mem1, mem2, mem3, mem4, and mem5; When the first row of data arrives, read out the data of mem1, mem2, mem3, mem4 corresponding to the 4 columns, and write the newly arrived 4 pixel values ​​into mem5; When the second row of pixel data arrives, read out the data of mem2, mem3, mem4, mem5 corresponding to the four columns, and write the new data into mem1; When the third row of data arrives, read out the data of mem3, mem4, mem5, mem1 corresponding to the four columns, and write the newly arrived four pixel values ​​into mem2; When the fourth row of pixel data arrives, read out the data of mem4, mem5, mem1, mem2 corresponding to the four columns, and write the new data into mem3; When the fifth row of data arrives, read out the data of mem5, mem1, mem2, and mem3 corresponding to the four columns, and write the newly arrived four pixel values ​​into mem4; When the sixth row of pixel data arrives, the data of mem1, mem2, mem3, and mem4 corresponding to the four columns are read out as in the first row, and the new data is written into mem5; And so on, all data are read in and out.

8. An electronic device, comprising a processor and a memory connected to the processor for storing instructions executable by the processor, characterized in that: The processor is used to execute the dynamic bad pixel correction method capable of reducing edge blur and lowering power consumption area as described in any one of claims 1-7 above.

9. A server, characterized in that: It includes at least one processor and a memory that is communicatively connected to the processor, wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the processor so that the at least one processor executes the dynamic bad pixel correction method that can reduce edge blur and reduce power consumption area as described in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the dynamic bad pixel correction method capable of reducing edge blur and lowering power consumption area as described in any one of claims 1 to 7 is implemented.

Images