A blind flash detection and compensation method based on spatiotemporal characteristics

Abstract

The invention discloses a blind flash pixel detection and compensation method based on time-space characteristics, and the specific process is as follows: step S1: for infrared images taken by ground-based telescopes, blind pixels in the image are detected by using the characteristics that the grayscale value of the blind pixel is basically unchanged in the image sequence and different from the normal pixel; step S2: the blind pixel grayscale value is replaced by the mean value of the eight neighborhoods of the blind pixel; step S3: the sequence maximum image is obtained for the image sequence with the blind pixel removed, and the position of the flash pixel in the sequence image is detected by using the characteristics that the grayscale value of the flash pixel in the sequence maximum image is jump-changed compared with the neighboring pixel, and the grayscale value of the flash pixel in the sequence maximum image should be the maximum value of its nine neighborhoods; step S4: for the detected flash pixel, the grayscale value when it appears as a normal pixel in the K-frame image sequence is counted, and the grayscale value of the flash pixel when it appears as a bad pixel in the image sequence is replaced by their mean value. The invention can avoid suppressing the target signal while detecting the blind flash pixel.

Description

Technical Field

[0001] The invention belongs to the field of ground-based astronomical observation and space detection, and in particular relates to a blind flash element detection and compensation method based on time and space characteristics. Background Art

[0002] Blind pixels are divided into overheated pixels and dead pixels. Overheated pixels refer to pixels whose noise voltage is greater than 10 times the average noise voltage ([1] Liu Zhenlong. Research on infrared image non-uniformity correction and enhancement technology [D]. Harbin Engineering University, 2013.), and dead pixels refer to pixels whose output response rate is lower than 1 / 10 of the average output response rate ([2] Zhao Zhennan, Song Hongfei, Ren Hongkai. An improved scene-based non-uniformity correction method [J]. Journal of Changchun University of Science and Technology (Natural Science Edition), 2020, 43(02): 53-57.). They appear as bright spots or dark spots in the image and are easily misdetected as targets. Flash pixels do not always appear as bad pixels in the image sequence ([3] Liu Gaorui, Sun Shengli, Lin Changqing, Lv Pingle. Analysis and suppression method of flash pixel noise in infrared array detectors [J]. Journal of Infrared and Millimeter Waves, 2018, 37(04): 421-426+432.). When it appears as a bad pixel, it appears as a bright spot in the image; and when it appears as a normal pixel, it is the same as other normal pixels. Therefore, the flickering pixel appears as a constantly flickering point in the image and is easily misdetected as a target.

[0003] The reason for the blind pixels is that due to defects in production process and materials, the infrared detector will randomly generate some blind pixels and flicker pixels. The pixels in the infrared detector that appear as blind pixels and flicker pixels are completely or seriously failed, causing their response to infrared radiation to deviate significantly from the normal value.

[0004] Although the time domain mean outlier extraction blind pixel detection algorithm (TMOE) can detect blind pixels, the TMOE algorithm will misdetect the target as a blind pixel. Therefore, after blind pixel compensation, the target will be suppressed, increasing the difficulty of detecting weak targets.

[0005] Traditional flicker compensation algorithms include neighborhood median compensation and neighborhood mean compensation. Under normal circumstances, these two algorithms can compensate flicker well. However, these two methods ignore the useful information that flicker may carry when it appears as a normal pixel. Therefore, when the pixel where the target is located appears as a flicker, these two flicker compensation algorithms will suppress the target, making target detection more difficult. Summary of the invention

[0006] In order to overcome the shortcomings of the existing solutions, the present invention provides a blind flash element detection and compensation method based on spatiotemporal characteristics, which is used to avoid affecting the detection of small targets while detecting and compensating blind flash elements. The method overcomes the defect that the TMOE algorithm will misdetect the target as a blind element, makes full use of the characteristics of the blind flash element that is different from the small targets, and ensures that the small targets are not misdetected or suppressed while detecting and compensating the blind flash element.

[0007] The technical solution of the present invention is a blind flash element detection and compensation method based on spatiotemporal characteristics, and its implementation steps include:

[0008] Step S1: for infrared images taken by ground-based telescopes, blind pixels in the image are detected by using the characteristics that the grayscale value of blind pixels is basically unchanged in the image sequence and different from that of normal pixels;

[0009] Step S2: Use the eight-neighborhood mean of the blind pixel to replace the blind pixel grayscale value;

[0010] Step S3: Find the maximum value image of the sequence of images with blind pixels removed, and detect the position of the flash pixel in the sequence image by using the characteristics that the gray value of the flash pixel in the maximum value image is jumpy compared with the neighboring pixels, and the gray value of the flash pixel in the maximum value image should be the maximum value of its nine neighboring pixels;

[0011] Step S4: For the detected flash pixel, count its grayscale value when it appears as a normal pixel in the K-frame image sequence, and use their mean value to replace the grayscale value of the flash pixel when it appears as a bad pixel in the image sequence.

[0012] The principle of the present invention is that the grayscale value of the blind pixel in the sequence image is basically unchanged, the grayscale value of the flash pixel has a jump, and the blind flash pixel has a significant difference from the neighboring pixel in the airspace. The blind flash pixel detection and compensation method based on the spatiotemporal characteristics of the present invention makes full use of the characteristics that the grayscale value of the blind pixel is basically unchanged in the image sequence, and different from the normal pixel, the grayscale value of the flash pixel has a jump compared with the neighboring pixel, which greatly reduces the false alarm rate of weak target detection and improves the detection rate of weak target detection.

[0013] The advantages of the present invention compared with the prior art are:

[0014] (1) The blind flash pixel detection and compensation algorithm based on spatiotemporal characteristics of the present invention overcomes the defect that the traditional blind pixel detection algorithm may misdetect the target as a blind pixel, thereby improving the detection probability of the target.

[0015] (2) When the pixel where the target is located appears as a flash pixel, the blind flash pixel detection and compensation algorithm based on the spatiotemporal characteristics of the present invention can retain the information when the pixel appears as a normal pixel, thereby overcoming the defect that the traditional flash pixel compensation algorithm will suppress the target.

[0016] (3) The infrared target image processing of the present invention adopts a blind flash element detection and compensation algorithm based on spatiotemporal characteristics, which can accurately and efficiently detect and compensate for blind flash elements in infrared images, while ensuring the target detection effect and reducing the false alarm rate of weak target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flow chart of a blind flash element detection and compensation method based on spatiotemporal characteristics of the present invention;

[0018] Figure 2a A schematic diagram of an original image including a blind flash element of the present invention;

[0019] Figure 2b The three-dimensional image of the original image of the present invention;

[0020] Figure 3 It is a schematic diagram of the blind pixel detection result of the present invention;

[0021] Figure 4a It is a schematic diagram of the blind pixel compensation result of the present invention;

[0022] Figure 4b A three-dimensional diagram of the blind pixel compensation result of the present invention;

[0023] Figure 5a It is a schematic diagram of a sequence maximum image of the present invention;

[0024] Figure 5b It is a three-dimensional image of the sequence maximum value of the present invention;

[0025] Figure 6 It is a schematic diagram of the flash element detection result of the present invention;

[0026] Figure 7a It is a schematic diagram of the flash element compensation result of the present invention;

[0027] Figure 7b A three-dimensional diagram of the flash element compensation result of the present invention; DETAILED DESCRIPTION

[0028] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0029] like Figure 1 As shown, the blind flash detection and compensation method based on spatiotemporal characteristics of the present invention is suitable for reducing the influence of blind flash of infrared detector on the detection of weak targets when ground-based telescopes shoot targets. Specifically, the blind flash detection and compensation method based on spatiotemporal characteristics includes the following steps:

[0030] Step S1: for infrared images taken by ground-based telescopes, blind pixels in the image are detected by using the characteristics that the grayscale value of blind pixels is basically unchanged in the image sequence and different from that of normal pixels;

[0031] Step S2: Use the eight-neighborhood mean of the blind pixel to replace the blind pixel grayscale value;

[0032] Step S3: Obtain the sequence maximum image for the image sequence after blind pixel removal, and use the characteristics that the gray value of the flash pixel in the sequence maximum image jumps compared with the neighboring pixels, and the gray value of the flash pixel in the sequence maximum image should be the maximum value of its nine neighboring pixels, to detect the position of the flash pixel in the sequence image.

[0033] Step S4: For the detected flash pixel, count its grayscale value when it appears as a normal pixel in the K-frame image sequence, and use their mean value to replace the grayscale value of the flash pixel when it appears as a bad pixel in the image sequence.

[0034] The process of the method of the present invention is as follows Figure 1 Specifically, the blind pixel detection and compensation algorithm of the present invention includes the following steps:

[0035] 1. The original infrared image is as follows Figure 2a As shown, take K frames of continuous infrared images and perform time domain averaging to obtain the average image

[0036]

[0037] Where I(i,j,k), k=1,2,…,K, represents the gray value of the pixel located in the i-th row and j-th column in the k-th frame image.

[0038] 2. Average image The edge of the image is mirrored and extended. China-Israel Take a window of size s×s as the center and calculate the grayscale median of the pixels in the window, denoted as i1=i+s, j1=j+s.

[0039] 3. Calculate pixels The absolute value of the difference between the grayscale time domain mean and the grayscale spatial domain median, that is, the outlier D(i,j):

[0040]

[0041] 4. Set a reasonable threshold Thr. If D(i,j)>Thr, proceed to step 5. Otherwise, the pixel I(i,j) is determined to be a normal pixel and return to step 2. Set Thr to:

[0042] Thr=μ D +tσD (3)

[0043] In the formula, μ D Represents the average value of all pixel wild values ​​in a single frame image, σ D represents the standard deviation of outliers, and t is a constant.

[0044]

[0045]

[0046] Where M and N represent the total number of rows and columns of a single frame image, respectively.

[0047] 5. When a pixel meets the threshold judgment condition in step 4, count the total number of frames in the K-frame image sequence where the grayscale value of the pixel is the maximum value of its nine neighborhoods, recorded as z. Set the blind pixel judgment condition as follows:

[0048]

[0049] 6. The blind pixel detection results are as follows Figure 3 As shown, Figure 3 The pixels marked by both the black circular frame and the white rectangular frame are the detected blind pixels. Only the pixels marked by the black circular frame meet the judgment conditions in step 4, but do not meet the judgment conditions in step 5. They may be targets, flash pixels, and normal pixels. For the pixels that have been detected as blind pixels, calculate the grayscale mean of its eight neighborhoods in each frame of the image, and use the mean to replace the grayscale value of the blind pixel in the frame of the image to obtain the image sequence I1 that completes blind pixel detection and compensation. The blind pixel compensation results are shown in Figure 4a shown.

[0050] The flash element detection and compensation algorithm of the present invention comprises the following steps:

[0051] 1. To avoid the failure of flash detection due to excessive fluctuation of sky light background, first calculate the background mean m of each frame in the K-frame image sequence I1. B (k):

[0052]

[0053] For m B (k) Sort by small to large, select the 20 frames of images in the middle, record them as I2(k1), and record the background mean of these 20 frames as m B1 (k1), k1=1,2,…,20.

[0054] 2. Find the maximum grayscale value of each pixel in I2 in 20 frames of images and obtain the sequence maximum image I max1 ,like Figure 5a shown.

[0055] 3. Put I max1 The edge of is mirrored and extended by 1 row and 1 column to obtain image I max2 .

[0056] 4. Traverse image I from top to bottom and from left to right max2 . to I max2 Pixel I in max2 (i2, j2), i2 = i + 1, j2 = j + 1, find the maximum grayscale value t1 of its nine neighborhoods. max2 If the gray value of (i2, j2) is equal to t1, proceed to step 5; otherwise, judge the next pixel.

[0057] 5. Calculate pixel I max2 The difference between the grayscale values ​​of (i2, j2) and its eight neighboring pixels is summed and averaged to obtain the pixel I max2 The gray value of (i2, j2) and the average change v(i, j) of the gray value of each pixel in its eight neighborhoods:

[0058]

[0059] In the formula, I s (j1) represents pixel I max2 The grayscale values ​​of the eight neighborhood pixels of (i2,j2), j1=1,2,…,8.

[0060] 6. Calculate pixel I max2 The grayscale mean v1 of the eight neighborhoods of (i2,j2):

[0061]

[0062] 7. Find m B1 The mean m1 and standard deviation σ1 of (k):

[0063]

[0064]

[0065] 8. Set the flash element judgment conditions:

[0066] when

[0067] When σ1>3, the sky light background changes dramatically, and the parameter r is set to 1; when σ1<0.8, the sky light background changes little, and the parameter r is set to 5; when 0.8≤σ1≤3, the sky light background changes relatively slowly, and the parameter r is set to σ1.

[0068] The flash test results are as follows Figure 6 As shown in the figure, the pixels marked by white circular frames are flash pixels.

[0069] 1. For the pixel I1(x,y) detected as a flash pixel in the above steps, the following judgments are made on the K-frame image sequence in sequence:

[0070] when

[0071] The grayscale values ​​of the pixels I1(i,j) in the K-frame image when they are normal pixels are counted, and their mean is used to replace the grayscale values ​​of the pixels I1(i,j) when they are bad pixels. The flash compensation results are shown in Figure 7a shown.

[0072] In order to compare the superiority of the flash compensation algorithm of the present invention over the traditional algorithm, when the pixel where the target is located is a flash pixel, the algorithm of the present invention, the neighborhood mean compensation algorithm and the neighborhood median compensation algorithm are used to perform blind flash pixel detection and compensation on the same image sequence, and then the image sequence is accumulated for multiple frames. The mean signal-to-noise ratio (MSNR) of the target after accumulation is shown in the following table:

[0073] Table 1. Comparison of target MSNR accumulated over multiple frames after being processed by the algorithm of the present invention and the traditional algorithm

[0074]

[0075] It can be seen from Table 1 that when the pixel where the target is located appears as a flash pixel, under the same cumulative number of frames, the algorithm of the present invention can obtain the highest target mean signal-to-noise ratio, indicating that the algorithm of the present invention has the least suppression on the target signal when compensating for the flash pixel, which can reduce the occurrence of target missed detection and improve the detection rate of target detection.

Claims

1. A blind flash detection and compensation method based on spatiotemporal characteristics, characterized in that: The method implementation steps include: Step S1: for infrared images taken by ground-based telescopes, blind pixels in the image are detected by using the characteristics that the grayscale value of blind pixels is basically unchanged in the image sequence and different from that of normal pixels; Step S2: Use the eight-neighborhood mean of the blind pixel to replace the blind pixel grayscale value; Step S3: Find the maximum value image of the sequence of images with blind pixels removed, and detect the position of the flash pixel in the sequence image by using the characteristics that the gray value of the flash pixel in the maximum value image is jumpy compared with the neighboring pixels, and the gray value of the flash pixel in the maximum value image should be the maximum value of its eight neighbors; Step S4: for the detected flash pixel, count its grayscale value when it appears as a normal pixel in the K-frame image sequence, and replace the grayscale value of the flash pixel when it appears as a bad pixel in the image sequence with their mean value; The flash element detection steps in step S3 are as follows: Step 41: Calculate the background mean m of each frame in the image sequence I1 after blind pixel detection and compensation B (k): (7) For m B (k) Sort by small to large, select the 20 frames of images in the middle, record them as I2(k1), and record the background mean of these 20 frames as m B1 (k1), k1=1,2,…,20; M, N represent the total number of rows and columns of a single frame image respectively; Step 42: Find the maximum grayscale value of each pixel in I2 in the 20 frames of images to obtain the sequence maximum image I max1 ; Step 43: I max1 The edge of is mirrored and extended by 1 row and 1 column to obtain image I max2 ; Step 44: Traverse image I from top to bottom and from left to right max2 , to I max2 Pixel I in max2 (i, j), find the maximum grayscale value t1 of its eight neighborhoods. If pixel I max2 If the gray value of (i, j) is equal to t1, proceed to step 45; otherwise, make a judgment on the next pixel; Step 45: Calculate pixel I max2 The difference between the grayscale values ​​of pixels (i, j) and its eight neighboring pixels is summed and averaged to obtain the pixel I. max2 The gray value of (i, j) and the average change v(i, j) of the gray value of each pixel in its eight neighborhoods: (8) In the formula, I s (j1) represents pixel I max2 Grayscale values ​​of the eight neighboring pixels of (i,j), j1=1,2,…,8; Step 46: Calculate pixel I max2 The grayscale mean v1 of the eight neighborhoods of (i,j): (9) Step 47: Find m B1 The mean m1 and standard deviation σ1 of (k1) are: (10) (11) Step 48: Set the flash element judgment conditions: (12) When σ1>3, the skylight background changes dramatically, and the parameter r is set to 1; when σ1<0.8, the skylight background changes little, and the parameter r is set to 5; when 0.8≤σ1≤3, the skylight background changes slowly, and the parameter r is set to σ1; Step 49: For the pixel I1(i,j) detected as a flash pixel in the above steps, the following judgments are made on the K-frame image sequence in sequence: (13)。 2. The blind flash detection and compensation method based on spatiotemporal characteristics according to claim 1, characterized in that: The blind pixel detection steps in step S1 are as follows: Step 21: Take K frames of continuous infrared images, perform time domain averaging, and obtain the average image : (1) In the formula, , k=1,2,…,K, represents the gray value of the pixel located in the i-th row and j-th column in the k-th frame image; Step 22: Average the images The edge of the image is mirrored and extended. China-Israel Take a window of size s×s as the center and calculate the grayscale median of the pixels in the window, denoted as ; Step 23: Calculate pixels The absolute value of the difference between the grayscale time domain mean and the grayscale spatial domain median, that is, the outlier D(i,j): (2) Step 24: Set a reasonable threshold Thr. If D(i,j)>Thr, proceed to step 25. Otherwise, the pixel I(i,j) is determined to be a normal pixel, and return to step 22 to set Thr to: (3) In the formula, μ D Represents the average value of all pixel wild values ​​in a single frame image, σ D represents the standard deviation of outliers, and t is a constant; (4) (5) In the formula, M and N represent the total number of rows and columns of a single frame image respectively; Step 25: When a pixel satisfies the threshold judgment condition in step 24, count the total number of frames in the K-frame image sequence where the grayscale value of the pixel is the maximum value of its eight neighborhoods, denoted as z, and set the blind pixel judgment condition as follows: (6)。 3. The blind flash detection and compensation method based on spatiotemporal characteristics according to claim 1, characterized in that: The blind pixel compensation steps in step S2 are as follows: Step 31: Calculate the grayscale mean of the eight neighborhoods of the blind pixel in each frame of image, and use the mean to replace the grayscale value of the blind pixel in the frame of image.

4. The blind flash detection and compensation method based on spatiotemporal characteristics according to claim 1, characterized in that: The flash compensation steps in step S4 are as follows: Step 51: Count the grayscale values ​​of the pixels I1(i, j) in the K-frame image when they appear as normal pixels, and replace the grayscale values ​​of the pixels I1(i, j) when they appear as bad pixels with their mean values.

Images