A method and device for processing dynamic bad points

Abstract

The present invention relates to the field of image processing technology, and solves the technical problem that the processing effect of some dynamic bad pixels is poor, resulting in poor imaging quality. In particular, it relates to a method for processing dynamic bad pixels, including: according to the characteristics of a color filter matrix, taking the central pixel of the matrix as the current pixel, and taking n pixels of the same color around the current pixel to form a reference array; calculating the average value of normal pixels in the reference array; taking the current pixel as the center and correspondingly dividing it into four adjacent pixel groups according to four directions, finding the direction of the edge according to the same color difference of each adjacent pixel group; judging whether the current pixel is a bad pixel according to the direction difference. The present invention determines the edge direction of the image content according to the color filter matrix and the color difference constant theory, finds out the abnormal pixels according to the change trend of the edge pixels, and then processes them. It is a bad pixel processing method based on real-time changes in image content, and improves the imaging quality.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and device for processing dynamic bad pixels. Background Art

[0002] Due to differences in semiconductor manufacturing processes and raw materials, the manufactured image sensors will inevitably have some bad pixels. The so-called bad pixels refer to pixels that always show the same brightness and do not change with the sensitivity. There are two main types of bad pixels: one is a static bad pixel where the pixel is always bright or dark; the other is a dynamic bad pixel where the difference in pixel value change over time is much smaller than that of the surrounding pixels. The existence of bad pixels is one of the reasons for the decline in image quality. Therefore, processing bad pixels is very necessary to improve image quality.

[0003] At present, in the prior art, the processing of bad pixels of image sensors is usually based on the deficiencies of the equipment. After the bad pixels of the image sensor are first found under specific conditions, the addresses of the detected bad pixels are stored in a bad pixel table, and the bad pixels are stored in a memory in a table form, and then the memory is called to solve the known bad pixel problem. However, the above-mentioned bad pixel processing method only directly processes the bad pixels in a simple manner, and there is no effective solution for some dynamic bad pixels. Therefore, the processing effect is poor, which affects the imaging quality and cannot meet people's use requirements in work and life. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides a method and device for processing dynamic bad pixels, which solves the technical problem that the processing effect of some dynamic bad pixels is poor, resulting in poor imaging quality, and achieves the purpose of real-time bad pixel processing based on changes in image content, improving imaging quality and being highly inclusive.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: a method for processing dynamic bad pixels, comprising the following processes:

[0006] S1. According to the characteristics of the color filter matrix, the center pixel of the matrix is ​​taken as the current pixel, and n pixels of the same color around the current pixel are taken to form a reference array;

[0007] S2. Calculate the average value of normal pixels in the reference array;

[0008] S3. The current pixel is centered on four adjacent pixel groups corresponding to the horizontal, vertical, 45° and 135° directions, and the edge direction is found according to the same color difference of each adjacent pixel group, and the corresponding color difference value is calculated;

[0009] S4. Determine whether the current pixel is a bad pixel based on the directional difference. If the maximum directional difference is greater than K times the minimum directional difference, and the minimum directional difference is greater than a set threshold, the current pixel is determined to be a bad pixel and the original pixel value of the current pixel is replaced by the correction value. Otherwise, the current pixel is determined to be a normal pixel and the pixel value of the current pixel is output.

[0010] Furthermore, before step S1, the method further includes:

[0011] S0. Obtain an image and define the pixel values ​​of all pixels in the image as a two-dimensional array, set a corresponding tag value for each pixel according to the detection result, and store the tag value of each pixel in a table in order.

[0012] Further, the correction value is determined according to whether the edge difference is valid. If the edge difference is valid, the correction value is a reference value on the boundary; otherwise, the correction value is the average value.

[0013] Furthermore, the color difference values ​​include the maximum direction difference value and the minimum direction difference value in four directions: horizontal, vertical, oblique 45° and oblique 135°.

[0014] A dynamic bad pixel processing device, the device comprising:

[0015] A selection unit, wherein the selection unit is used to take the center pixel of the matrix as the current pixel and take n pixels of the same color around the current pixel to form a reference array according to the characteristics of the color filter matrix;

[0016] A first calculation unit, the first calculation unit is used to calculate an average value of normal pixels in the reference array;

[0017] A second calculation unit, the second calculation unit is used to divide the current pixel into four adjacent pixel groups according to the four directions of horizontal, vertical, oblique 45° and oblique 135°, find the direction of the edge according to the same color difference of each adjacent pixel group, and calculate the corresponding direction difference;

[0018] A judgment and correction unit is used to judge whether the current pixel is a bad pixel according to the directional difference. If the maximum directional difference is greater than K times the minimum directional difference, and the minimum directional difference is greater than a set threshold, the current pixel is judged to be a bad pixel and the original pixel value of the current pixel is replaced by the correction value. Otherwise, the current pixel is judged to be a normal pixel and the pixel value of the current pixel is output.

[0019] Furthermore, the device also includes an image acquisition unit, which is used to acquire an image and define the pixel values ​​of all pixels in the image as a two-dimensional array, set a corresponding label value for each pixel according to the detection result, and store the label value of each pixel in sequence in a table.

[0020] By means of the above technical solution, the present invention provides a method and device for processing dynamic bad pixels, which have at least the following beneficial effects:

[0021] 1. The present invention determines the edge direction of the image content according to the color filter matrix and the color difference constancy theory, finds out the abnormal pixels according to the change trend of the edge pixels and then processes them. It is a bad pixel processing method based on the real-time change of image content, and also includes the processing of some obvious noise points, thereby improving the image processing effect and being easy to promote and apply.

[0022] 2. The present invention shifts the fixed-point processing based on the device to dynamic bad pixel processing based on the image content, which effectively solves the problem of correcting dynamic bad pixels. It does not distinguish between bright spots and dark spots and has better inclusiveness, thus being a better supplement to static bad pixel processing and having higher market value and application prospects.

[0023] The present invention determines the edge direction of the image content according to the color filter matrix and the color difference constancy theory, and finds out the abnormal pixels according to the change trend of the edge pixels and then processes them. It is a bad pixel processing method based on real-time changes in image content, and also includes the processing of some obvious noise points. It has the advantages of strong inclusiveness and good processing effect, has high market value and application prospects, and is easy to promote and apply. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0025] Figure 1 It is a flow chart of the method for processing dynamic bad pixels of the present invention;

[0026] Figure 2 A schematic diagram of pixels of the same color in a RAW image involved in the method for processing dynamic bad pixels of the present invention;

[0027] Figure 3 Schematic diagrams of four directions in a RAW image involved in the method for processing dynamic bad pixels of the present invention;

[0028] Figure 4 It is a block diagram of the dynamic bad pixel processing device of the present invention. DETAILED DESCRIPTION

[0029] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods, so that the implementation process of how the present application uses technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly.

[0030] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, so the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0031] Please refer to Figure 1-Figure 4 , shows a method for processing dynamic bad pixels according to the first embodiment of the present invention. In this embodiment, the image to be processed is in RAW image format, and specifically includes the following steps:

[0032] S0. Get a RAW image and define the pixel values ​​of all pixels in the image as a two-dimensional array RawImg, where RawImg(y,x) represents the current pixel, that is, the yth row and xth column in the image. Set a corresponding tag value for each pixel according to the detection result, and store the tag value of each pixel in the table in order.

[0033] S1. In the RAW image, according to the characteristics of the color filter matrix (CFA), the center pixel of the matrix is ​​taken as the current pixel, such as Figure 2 As shown in c, n pixels of the same color around the current pixel are taken to form a reference array. The pixel values ​​of the reference array are defined as another two-dimensional array BlockBuf;

[0034] In this embodiment, n=8, and they are recorded as s0~s7 respectively. At the same time, the known bad pixel mark values ​​stored in the memory are located as a two-dimensional array BlockDPs, corresponding to the positions of s0~s7, respectively marked as sd0~sd7. For example, when sd0=1, it means that s0 is known to be marked as a bad pixel. If sd0=0, it means that s0 is a normal pixel.

[0035] S2. Calculate the average value of normal pixels in the reference array. The calculation formula is as follows:

[0036]

[0037] S3. Taking the current pixel as the center, the pixels are divided into four adjacent pixel groups according to the horizontal, vertical, 45° and 135° directions respectively. The direction of the edge is found according to the same color difference of each adjacent pixel group, and the direction difference in the four directions is calculated. The specific calculation process is as follows:

[0038] 1) Calculate the average value of the same color pixels on both sides of the current pixel in the horizontal, vertical, 45° and 135° directions, the absolute value of the difference, and the absolute value of the difference between the current pixel and the average value:

[0039]

[0040] p2_diff(i)=|s i -s i+4 |

[0041] diff_p2avg(i)=|c-p2_avg(i)|

[0042] 2) Determine the smoothness of the horizontal, vertical, 45° and 135° directions one by one. For example, if it is known that there are no bad pixels, that is, (sd i +sd i+4 = = 0), if the absolute value of the difference between the current pixel and the average value is less than half of the absolute value of the difference between two adjacent pixels in the same direction, that is, It is considered that the gradient change in this direction is smooth, and flag_smth[i]=1 is marked, otherwise flag_smth[i]=0, and the edge difference in this direction is marked as diff_p2dir[i]=p2_diff(i). If it is not smooth, diff_p2dir[i]=255 is marked.

[0043] 3) In the four directions of horizontal, vertical, oblique 45° and oblique 135°, based on the premise that the vertical direction is not smooth, find the direction with the smallest edge difference, and record the average value in this direction as the reference value ref_edge=p2_avg(i) on the edge; at the same time, if the difference in the current edge direction diff_p2dir(i)≤MAX(16,blk_avg / 4), the flag flag_p2dir that the current edge difference is valid is set to 1 (it should be noted that the initialization value of flag_p2dir is 0).

[0044] 4) Calculate the maximum directional difference, as well as the minimum and maximum values ​​of the average value, to determine whether to use the edge reference value ref_edge during bad pixel correction:

[0045]

[0046] diff_p2avg_min=min 0≤k≤3 diff_p2avg(k)

[0047]

[0048] S4. Determine whether the current pixel is a bad pixel according to the direction difference. The specific process is as follows:

[0049] 1) If there are no known bad pixels in the horizontal, vertical, 45° and 135° directions, calculate the change trend of the same color pixels and the color difference change trend, and extract the minimum difference as the possible edge difference.

[0050] In this embodiment, in order to facilitate calculation, the original pixel values ​​in BlockBuf and the bad pixel marks in BlockDPs are sorted by direction, such as Figure 3 As shown, the array line_pix(0,0~4) represents the five pixels A, B, C, D and E in the horizontal direction; similarly, line_pix(1,0~4) represents the five pixels at an angle of 135°, line_pix(2,0~4) represents the five pixels in the vertical direction, and line_pix(3,0~4) represents the five pixels at an angle of 45°.

[0051] line_sdp(0,0~4) indicates the bad pixel mark of five pixels in the horizontal direction. If the pixel has been defined as a bad pixel, it is marked as 1, otherwise it is marked as 0. Similarly, line_sdp(1,0~4) indicates the bad pixel mark of five pixels at an angle of 135°, line_sdp(2,0~4) indicates the bad pixel mark of five pixels in the vertical direction, and line_sdp(3,0~4) indicates the bad pixel mark of five pixels at an angle of 45°.

[0052] 2) In the horizontal, vertical, 45° and 135° directions, count the first four consecutive pixels and the last four consecutive pixels, and count the number of known bad pixels.

[0053] For example, in the horizontal direction It can be seen that a total of 8 statistical values ​​are generated in the four directions:

[0054]

[0055]

[0056] Among them, 0≤i<4, indicating four directions.

[0057] In each direction, the first four pixels in line_pix are taken as the left pixels, and the last four pixels are taken as the right pixels. For example, in the horizontal direction, line_pix(0,0~3) represents the left side, line_pix(0,1~4) represents the right side, and so on.

[0058] If there is a bad pixel in the four pixels on the left (sum_lsdp(k,0) == 1), set line_ydif_lft(i) to the maximum value of 255, otherwise calculate the absolute value of the difference between the current pixel and the predicted pixel value in the left pixel in each direction:

[0059] line_ydif_lft(i)=|line_pix(i,2)-(line_pix(i,0)+line_pix(i,3)-line_pix(i,1))|

[0060] And the minimum absolute value of the difference between pixels of the same color:

[0061] line_cdif_lft(i)=min(|line_pix(i,0)-line_pix(i,2)|,|line_pix(i,1)-line_pix(i,3)|)

[0062] And when the condition (line_cdif_lft(i)>MAX_VAL*3 / 4) is met, line_ydif_lft(i) needs to be reset to the maximum value 255.

[0063] Similarly, if there is a bad pixel in the four pixels on the right (sum_lsdp(k,1) == 1), then line_ydif_rgt(i) is set to the maximum value 255, otherwise the following calculation is performed:

[0064] line_ydif_rgt(i)=|line_pix(i,2)–(line_pix(i,4)+line_pix(i,1)–line_pix(i,3))|line_cdif_rgt(i)=min(|line_pix(i,1)-line_pix(i,3)|,|line_pix(i,2)-line_pix(i,4)|)

[0065] And when the condition (line_cdif_rgt(i)>M X_VAL*3 / 4) is met, line_ydif_rgt(i) needs to be reset to the maximum value 255.

[0066] It should be noted that MAX_VAL represents the maximum value of a pixel, and the maximum value varies depending on the bit width of the pixel data. For example, when 10 bits are used, MAX_VAL=1023.

[0067] 3) Calculate the difference in the horizontal, vertical, 45° and 135° directions:

[0068] line_ydif(i)=(min(line_ydif_lft(i),line_ydif_rgt(i))+diff_p2avg(i)) / 2

[0069] Among them, 0≤i<4, indicating four directions. The minimum direction difference can be obtained:

[0070]

[0071] 4) Among the four directional differences, if the largest directional difference is greater than K times the smallest directional difference, and the smallest directional difference is greater than the set threshold THD_LINE_DIF, the current pixel is determined to be a bad pixel and the original pixel value of the current pixel is replaced by the correction value; otherwise, the current pixel is determined to be a normal pixel and the pixel value of the current pixel is output.

[0072] like Figure 4 As shown, the present invention provides a technical solution: a dynamic bad pixel processing device, the device comprising:

[0073] The selection unit 100 is used to select the center pixel of the matrix as the current pixel according to the characteristics of the color filter matrix, and select n pixels of the same color around the current pixel to form a reference array;

[0074] A first calculation unit 200, the first calculation unit 200 is used to calculate an average value of normal pixels in a reference array;

[0075] The second calculation unit 300 is used to divide the pixels into four adjacent pixel groups according to the four directions of horizontal, vertical, oblique 45° and oblique 135° with the current pixel as the center, find the direction of the edge according to the same color difference values ​​of each adjacent pixel group, and calculate the color difference values ​​in the four directions;

[0076] The judgment correction unit 400 is used to judge whether the current pixel is a bad pixel according to the color difference value. If the maximum directional difference value is greater than K times the minimum directional difference value, and the minimum color difference value is greater than a set threshold, the current pixel is judged to be a bad pixel and the original pixel value of the current pixel is replaced by the correction value. Otherwise, the current pixel is judged to be a normal pixel and the pixel value of the current pixel is output.

[0077] The device also includes an image acquisition unit 500, which is used to acquire an image and define the pixel values ​​of all pixels in the image as a two-dimensional array, set a corresponding label value for each pixel according to the detection result, and store the label value of each pixel in sequence in a table.

[0078] In this embodiment, according to the color difference constant theory, the difference values ​​of adjacent pixels of the same color are equal. If the difference value between the current pixel and the adjacent pixels of the same color is greatly different from the difference value between the adjacent pixels of different colors, the current pixel may be a bad pixel. Then, the average value of the two adjacent pixels of the same color along the edge direction of the image is used as the correction value of the current pixel, thereby improving the efficiency and effect of image bad pixel processing.

[0079] The present invention determines the edge direction of the image content according to the color filter matrix and the color difference constancy theory, and finds out the abnormal pixels according to the change trend of the edge pixels and then processes them. It is a bad pixel processing method based on real-time changes in image content, and also includes the processing of some obvious noise points. It has the advantages of strong inclusiveness and good processing effect, has high market value and application prospects, and is easy to promote and apply.

[0080] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the above embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0081] The above implementation methods have been described in detail. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for processing dynamic bad pixels, characterized in that: The process includes: S1. According to the characteristics of the color filter matrix, the center pixel of the matrix is ​​taken as the current pixel, and the values ​​of n pixels of the same color around the current pixel are taken to form a reference array; S2. Calculate the average value of normal pixels in the reference array; S3. Divide the points into four groups of neighboring points in the horizontal, vertical, 45° and 135° directions, centering on the current pixel, and find the direction of the edge according to the color difference of the four neighboring point groups, and calculate the corresponding direction difference; S4. judging whether the current pixel is a bad pixel according to the directional difference; if the maximum directional difference is greater than K times the minimum directional difference, and the minimum directional difference is greater than a set threshold, the current pixel is judged to be a bad pixel and the original pixel value of the current pixel is replaced by the correction value; otherwise, the current pixel is judged to be a normal pixel and the pixel value of the current pixel is output; Before step S1, the method further includes: S0. Obtain an image and define the pixel values ​​of all pixels in the image as a two-dimensional array, set a corresponding tag value for each pixel according to the detection result, and store the tag value of each pixel in a table in order.

2. The method for processing a dynamic bad pixel according to claim 1, characterized in that: The correction value is determined according to whether the edge difference is valid. If the edge difference is valid, the correction value is a reference value on the boundary; otherwise, the correction value is the average value.

3. The method for processing a dynamic bad pixel according to claim 1, characterized in that: The direction difference includes the maximum direction difference and the minimum direction difference in four directions: horizontal, vertical, oblique 45° and oblique 135°.

4. A dynamic bad pixel processing device, characterized in that: The device includes: A selection unit (100), the selection unit (100) being used to, according to the characteristics of a color filter matrix, take a central pixel of the matrix as a current pixel, and take n pixel values ​​of the same color around the current pixel to form a reference array; A first calculation unit (200), the first calculation unit (200) is used to calculate an average value of normal pixels in a reference array; A second calculation unit (300), the second calculation unit (300) is used to divide the pixel into four adjacent point groups corresponding to the four directions of horizontal, vertical, oblique 45° and oblique 135°, with the current pixel as the center, find the direction of the edge according to the same color difference of the four adjacent point groups, and calculate the corresponding direction difference; A judgment correction unit (400) is used to judge whether the current pixel is a bad pixel according to the directional difference value; if the maximum directional difference value is greater than K times the minimum directional difference value, and the minimum directional difference value is greater than a set threshold value, the current pixel is judged to be a bad pixel and the original pixel value of the current pixel is replaced by the correction value; otherwise, the current pixel is judged to be a normal pixel and the pixel value of the current pixel is output.

5. The dynamic bad pixel processing device according to claim 4, characterized in that: The device also includes an image acquisition unit (500), which is used to acquire an image and define the pixel values ​​of all pixels in the image as a two-dimensional array, set a corresponding label value for each pixel according to the detection result, and store the label value of each pixel in sequence in a table.

Images