Infrared blind pixel detection method based on temperature self-adaption and time-frequency domain integration

Abstract

The invention relates to the field of infrared image processing, in particular to an infrared blind pixel detection method based on temperature self-adaption and time-frequency domain synthesis, which comprises the following steps of: traversing an infrared image by using a sliding window to obtain gray values of pixel points around a current pixel point; carrying out image edge judgment by using a Sobel-Otsu algorithm based on working temperature dynamic adjustment, and reserving a non-image edge window to participate in blind pixel judgment; grouping pixel points in the window according to horizontal and vertical directions; calculating the alpha filtering value of the gray scale of each group of pixels; calculating the deviation between the gray value of each pixel in the window and the alpha filtering value of the corresponding row coordinate and column coordinate group; if the deviation is greater than an adaptive threshold, setting the blind pixel as a potential blind pixel, and constructing a potential blind pixel distribution matrix; a time domain accumulation method for pre-screening periodic noise based on FFT is adopted, periodic noise interference is eliminated through frequency domain analysis, and then high-probability blind elements are screened through time domain accumulation; and obtaining a final blind pixel detection result. The method can effectively cope with the working temperature change of the sensor and the noise interference in a complex environment, and remarkably improves the accuracy and anti-interference capability of blind pixel detection.

Description

Technical Field

[0001] The present invention relates to the field of infrared image processing, and particularly to an infrared blind pixel detection method based on temperature self - adaptation and time - frequency domain synthesis. Background Art

[0002] Due to various factors such as technology, manufacturing materials, and environment, most of the infrared images output by infrared detectors have a certain number of blind pixels. Blind pixels will show over - bright points and over - dark points compared with effective pixels in imaging, interfering with the subsequent detection and recognition of targets and seriously affecting the imaging quality.

[0003] Existing blind pixel detection methods include the radiation calibration method and the method based on scene detection. The radiation calibration method is simple and conducive to hardware implementation. This method obtains the threshold of blind pixel detection gray level through high - and low - temperature blackbody imaging to achieve the purpose of detecting fixed blind pixels. The scene - based adaptive blind pixel detection method determines the pixels outside μ + 3σ of the gray value as blind pixels according to the characteristic that the gray value distribution of the window infrared image approximately satisfies the normal distribution.

[0004] At present, most of the blind pixel detections of infrared images use the scene - based adaptive blind pixel detection method, which adopts the median filtering algorithm. Although this kind of blind pixel detection method can achieve blind pixel detection in various situations, the detection result is greatly affected by the quality of infrared images. In actual application scenarios, if the detector is significantly disturbed by noise, the quality of the obtained infrared image will be significantly reduced. In addition, the change of environmental temperature will directly affect the working performance of the infrared detector, and thus affect the imaging quality. Therefore, the scene - based adaptive blind pixel detection method is prone to misdetection in complex working environments and needs to be improved. Summary of the Invention

[0005] To solve the defect that the scene - based adaptive blind pixel detection method is easily affected by the working temperature fluctuation of the infrared sensor and external noises (including periodic noise and Gaussian noise), resulting in a low accuracy of blind pixel detection, the present invention provides an infrared blind pixel detection method based on temperature self - adaptation and time - frequency domain synthesis, which can effectively cope with the change of the working temperature of the sensor and the noise interference in a complex environment, and significantly improve the accuracy and anti - interference ability of blind pixel detection.

[0006] To achieve the above object, the present invention provides an infrared sensor blind pixel detection method based on temperature self - adaptation and time - frequency domain synthesis, including the following steps:

[0007] S1. Take a (2n + 1)×(2n + 1) sliding window to traverse the infrared image, and obtain the gray values of (2n + 1) 2 pixels around the current pixel point, denoted as G(i,j), where (i,j) is the coordinate of the central pixel.

[0008] S2. Perform image edge judgment. If the image within the sliding window is determined to be a non-image edge after judgment, proceed to the next step; if it is determined to be an image edge, it does not participate in the blind pixel judgment.

[0009] S3. Group the pixel points extracted within the window in the horizontal direction (with a region size of (2n + 1)×1) and the vertical direction (with a region size of 1×(2n + 1)).

[0010] S4. According to the alpha filtering algorithm, first remove the maximum and minimum gray values of each group of pixel points, and then calculate the average of the remaining pixel gray values in each group, that is, the alpha filtering value H h (p) and H v (q), where H h (p) represents the filtering value of the p-th row group, and H v (q) represents the filtering value of the q-th column group.

[0011] S5. Calculate the absolute value of the gray value of each pixel within the window and the alpha filtering values of the pixel gray values corresponding to its row coordinates and column coordinates respectively as the judgment value for blind pixel detection. If the absolute value is greater than the adaptive threshold (the adaptive threshold is given by a threshold model adaptively adjusted based on temperature changes), set it as a potential blind pixel. The specific judgment criteria are as follows:

[0012] |G(p,q) - H h (p)| ≥ α

[0013] |G(p,q) - H v (q)| ≥ α

[0014] Where G(p,q) is the gray value of any pixel within the window, and α is the adaptive threshold given by a threshold model adaptively adjusted based on temperature changes.

[0015] S6. Use the time-domain accumulation method based on FFT pre-screening to remove periodic noise interference for secondary confirmation. First, exclude periodic noise interference through frequency-domain analysis, and then screen high-probability blind pixels through time-domain accumulation.

[0016] S7. Obtain the final blind pixel detection result.

[0017] Furthermore, the value of the sliding window in step S1 is one of the three values 1, 2, and 3.

[0018] Furthermore, the image edge judgment in step S2 includes the following steps:

[0019] S2.1 When the working environment temperature of the sensor is relatively high or low, the change in sensor noise will affect the edge sharpness of the image. Using a higher-order Sobel kernel can smooth the noise, but it will blur the image edges; while using a lower-order Sobel kernel can capture edge information more precisely, but it is vulnerable to noise. Therefore, the present invention provides an adjustable Sobel kernel that dynamically adjusts according to temperature intervals, which can dynamically adapt to different temperatures, reduce the impact of changes in the working temperature of the sensor on image edge discrimination, and ensure the accuracy of obtaining the gray gradient value by the Sobel kernel. Its dynamic adjustment method is as follows:

[0020]

[0021] S2.2. Obtain the gray values of each pixel of the image in turn through a sliding window of size δ×δ. The value of δ is the same as the order of the Sobel kernel corresponding to the temperature range of the current working environment.

[0022] S2.3. Normalize the gray values of the pixels in each window to the integer interval [0, 255]. This process is achieved through the following formula:

[0023]

[0024] where G(p,q) is the gray value of a certain pixel in the window before normalization, and G * (p,q) is the gray value of the pixel at the corresponding coordinate in the window after normalization, and max_Grey and min_Grey are the maximum and minimum values of the pixel gray levels in the window respectively.

[0025] S2.4. Perform a convolution multiplication operation on the gray values of each pixel in the window with the Sobel kernel. Here, taking δ = 5 as an example, the convolution factors in the horizontal direction and the vertical direction are respectively:

[0026]

[0027] Obtain the horizontal gradient G x and the vertical gradient G y .

[0028] S2.5. Calculate the gradient magnitude corresponding to this window. The calculation formula is as follows:

[0029]

[0030] For the convenience of hardware implementation, use the arithmetic mean of the absolute values to replace the square root result as follows:

[0031]

[0032] S2.6. Compare the G value with the set threshold. If it is greater than the set threshold, it is determined as the image edge.

[0033] The set threshold is given by the Otsu method based on gradient simplification. The traditional Otsu method is an adaptive threshold determination method. This method needs to traverse all possible gray values to calculate the between-class variance, which has a large computational amount in high-resolution images or real-time applications and needs to be improved.

[0034] Arrange the gradient amplitudes of each window calculated in S2.5 from large to small, and only take the gradients of the pixel points with the top 20% gradient amplitudes to participate in the traversal, narrowing the traversal threshold range, and perform the Otsu method operation on the gradient values within this range. The specific process is as follows:

[0035] The set threshold is given by the Otsu method based on gradient simplification. Arrange the gradient amplitudes of each window calculated in S3.5 from large to small, and only take the gradient values of the pixel points with the top 20% gradient amplitudes to participate in the traversal, and perform the Otsu method operation on the gradient values within this range. The specific process is as follows:

[0036] Suppose the gradient values participating in the traversal are in the interval [a, b], and there are m windows with the gradient value i, i ∈ [a, b]. i Then the probability of the window with gradient i appearing where M is the total number of windows participating in the traversal. The final ideal threshold β divides the windows into two parts: edge windows (gradient values in the range [β, b], denoted as T windows) and non-edge windows (gradient values in the range [a, β), denoted as F windows).

[0037] The probability p of the T window appearing T (denoted as a function f(β) with the threshold β as the independent variable) and the gradient mean μ T are respectively:

[0038]

[0039] The probability p of the F window appearing F and the gradient mean μ F are respectively:

[0040]

[0041] The total mean of the gray gradients of the image windows participating in the traversal:

[0042] μ = p T μ T + p F μ F

[0043] Between-class variance of two types of windows:

[0044] σ 2 (β) = p T (μ T - μ) 2 + p F (μ F - μ) 2

[0045] Then the finally set threshold β * is the β that satisfies the maximum of σ 2 (β), that is * , namely

[0046]

[0047] S2.7 Output the final edge detection result, and screen out the windows that are not at the image edge for blind pixel detection.

[0048] Furthermore, in step S4, the alpha filter values H h (p) and H v (q) are calculated by the following formula:

[0049]

[0050] where d is the number of pixels removed, d = 2, s represents the abscissa of any point in the q-th column of the window, and t represents the ordinate of any point in the q-th row of the window. G(p, t) is the gray value of all pixels in the p-th row within the window, and G(s, q) is the gray value of all pixels in the q-th column within the window.

[0051] Furthermore, the adaptive threshold required for the blind pixel detection decision in step S5 is given by a threshold model that adaptively adjusts based on temperature changes. This model includes the following steps:

[0052] S5.1 At the standard temperature T0, the blind pixel distribution is judged by the "3σ" method. Let μ0 represent the mean of the gray values of the pixel points within the window at the standard temperature, and σ0 represent the standard deviation of the gray values of the pixel points within the window at the standard temperature T0.

[0053] S5.2 When the temperature T changes, the threshold α also needs to change accordingly. The present invention establishes a model for adaptively adjusting the threshold with temperature changes. The influencing factors of the adaptive threshold are determined as: dark current I dark , signal-to-noise ratio SNR, and sensor hardware attributes.

[0054] Firstly, the relationship between the dark current I dark and the temperature T generally satisfies the Arrhenius equation:

[0055]

[0056] Before the device leaves the factory, multiple groups of data values are measured to obtain the proportionality coefficient ω of the device, that is:

[0057]

[0058] where k is the Boltzmann constant (k ≈ 8.617×10 -5 eV / K), E α is the activation energy, taking its general value of 0.7 eV, and ω is the proportionality coefficient.

[0059] Second, the signal-to-noise ratio SNR is the ratio of the signal power to the noise power, usually expressed in decibels (dB):

[0060]

[0061] P s is the image signal power, and P n is the thermal noise power. When the temperature T increases, the thermal noise of the device will increase accordingly, causing the signal-to-noise ratio SNR to decrease.

[0062] Third, the increase in temperature may also amplify the defects of the sensor itself and increase the probability of blind pixels.

[0063] The present invention uses the analytic hierarchy process to give the weights of the three influences. The judgment matrix A is listed as:

[0064]

[0065] According to the judgment matrix A, the weight vector W of the material itself defects, thermal noise, and dark current can be obtained as:

[0066] W = [0.11 0.58 0.31]

[0067] It can be seen from the final weight ratio that the influence of the material itself defects only accounts for 0.11, and its influence is ignored. Then the formula for the adaptive threshold α changing with temperature can be expressed as:

[0068]

[0069] where is the temperature rise and fall flag, T0 = 300K, SNR is the signal-to-noise ratio measured at temperature T, SNR0 is the standard signal-to-noise ratio measured at temperature T0, and I dark is the magnitude of the dark current measured at temperature T, and I dark0 is the standard dark current magnitude measured at temperature T0.

[0070] Furthermore, the time-domain accumulation method for pre-screening periodic noise based on FFT described in step S7 includes the following steps:

[0071] S7.1 Assume that N potential blind pixel distribution matrices D(p,q) are obtained in N frames. For each pixel (p,q) in the candidate blind pixel set, extract the time series G(p,q,n) of its grayscale value. * ),n * ∈[0,N],n * is the frame number, i.e. the nth frame in N frames * frame.

[0072] S7.2 For the time series G(p,q,n * )Use FFT transformation to get F(p,q,ω), that is:

[0073]

[0074] Get the maximum amplitude A of the non-zero frequency max (p,q):

[0075]

[0076] If the maximum amplitude A max If (p, q) is greater than or equal to the frequency domain threshold μ+3σ, the pixel is considered to be disturbed by periodic noise; otherwise, the pixel is considered to be not disturbed by noise, where μ and σ are all grayscale data values ​​G(p, q, n * ), n * is the frame number, i.e. the nth frame in N frames * frame.

[0077] S7.3 Sum the N potential blind pixel distribution matrices to obtain the cumulative value D sum (p,q), and compare it with the blind pixel detection threshold ρD max In comparison, if it is greater than the blind pixel detection threshold, it is determined to be the final blind pixel; otherwise, it is determined to be a normal pixel. Where ρ is the confidence level, which is usually 0.9, D max It is the largest element in the blind element matrix after the time domain accumulation of multiple frame images, that is:

[0078] D max =max{D sum (p,q)}

[0079] The final multi-frame image time domain cumulative blind pixel decision matrix is ​​recorded as DF(p,q), that is:

[0080]

[0081] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, the infrared blind pixel detection method based on temperature adaptation and time-frequency domain integration is implemented.

[0082] A computer-readable storage medium, on which computer instructions are stored, and when the computer instructions are executed by a processor, the infrared blind pixel detection method based on temperature adaptation and time-frequency domain synthesis as described above is implemented.

[0083] The beneficial effects of the present invention are:

[0084] (1) Strong anti-noise interference ability: Through the alpha mean filtering algorithm, it can effectively cope with the mixed interference of Gaussian noise and impulse noise, and significantly improve the accuracy of blind pixel detection. Through the time-domain accumulation algorithm, combined with frequency-domain analysis to exclude periodic noise interference, it can screen high-probability blind pixels in multiple consecutive frames of images, reduce the misjudgment rate, and improve the robustness of blind pixel detection.

[0085] (2) Strong adaptability: Using the Sobel-Otsu algorithm based on dynamic temperature adjustment for image edge decision-making can adapt to the sensor noise changes in different temperature environments, reduce the impact of temperature fluctuations on image edge discrimination, and improve the accuracy of edge detection.

[0086] (3) High computing and processing efficiency: Performing image edge pre-discrimination before blind pixel detection reduces the computational amount of blind pixel detection for the entire image, and at the same time reduces the possibility of misjudgment on the image edge, improving the real-time performance and computational efficiency of the algorithm.

[0087] (4) Strong dynamic adaptation ability: Through the adaptive threshold model based on temperature changes, it can dynamically adjust the blind pixel detection threshold according to the changes in the working temperature of the sensor, further improving the accuracy and adaptability of detection.

[0088] (5) Wide application range: It is applicable to infrared image processing in complex environments, can effectively cope with various noise interferences and temperature changes, and has a wide range of application prospects. Description of the Drawings

[0089] Figure 1 is a flowchart of one specific embodiment of the present invention.

[0090] Figure 2 is a flowchart of the blind pixel detection and confirmation algorithm of one specific embodiment of the present invention. Detailed Embodiments

[0091] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0092] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish one element from another.

[0093] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of this application.

[0094] Embodiment: As Figure 1 shown, an infrared blind pixel detection method based on temperature adaption and time-frequency domain synthesis specifically includes the following steps:

[0095] Take a (2n + 1)×(2n + 1) sliding window to traverse the infrared image, and obtain the gray values of (2n + 1) 2 pixel points around the current pixel point, denoted as G(i, j), where (i, j) is the coordinate of the central pixel. The value of the sliding window n is one of the three values 1, 2, and 3.

[0096] Perform image edge judgment: If the image within the sliding window is determined to be a non-image edge after judgment, then proceed to the next step; if it is determined to be an image edge, it does not participate in the blind pixel judgment.

[0097] When the operating temperature of the sensor is relatively high or low, the change in sensor noise will affect the edge sharpness of the image. Using a higher-order Sobel kernel can smooth the noise, but it will blur the image edges; while using a lower-order Sobel kernel can capture edge information more precisely, but it is easily affected by noise. The present invention provides an adjustable Sobel kernel that dynamically adjusts according to temperature intervals, which can dynamically adapt to different operating temperature environments, reduce the impact of changes in the sensor operating temperature on image edge discrimination, and ensure the accuracy of obtaining the gray gradient value by the Sobel kernel. Its dynamic adjustment method is shown in Table 1 below:

[0098]

[0099] Obtain the gray values of each pixel of the image in sequence through a sliding window of size δ×δ. The value of δ is the same as the order of the Sobel kernel corresponding to the temperature interval where the current temperature environment is located.

[0100] Normalize the gray values of the pixels in each window to the integer interval [0, 255]. This process is achieved through the following formula:

[0101]

[0102] Among them, G(p,q) is the gray value of a certain pixel in the window before normalization, and G * (p,q) is the gray value of the pixel at the corresponding coordinate in the window after normalization. max_Grey and min_Grey are respectively the maximum and minimum gray values of the pixels in the window.

[0103] This normalization process enables the gray values of the input image to be evenly distributed throughout the gray level range, thereby increasing the contrast and improving the decision-making accuracy.

[0104] Perform a convolution multiplication operation on the gray values of each pixel in the window with the Sobel kernel. Here, taking δ = 5 as an example, the convolution factors in the horizontal direction and the vertical direction are respectively:

[0105]

[0106] Obtain the horizontal gradient G x and the vertical gradient G y of each sampling window.

[0107] Calculate the gradient magnitude corresponding to this window. The calculation formula is as follows:

[0108]

[0109] For the convenience of hardware implementation, use the arithmetic mean of the absolute values to replace the square root result as follows:

[0110]

[0111] Compare the G value with the set threshold. If it is greater than the set threshold, it is judged as the image edge.

[0112] The set threshold is given by the Otsu method based on gradient simplification. The traditional Otsu method is an adaptive threshold determination method. This method needs to traverse all possible gray values to calculate the between-class variance, and the computational complexity is large in high-resolution images or real-time applications, and improvements are needed.

[0113] Arrange the gradient magnitudes of each window calculated in the previous step from large to small, and only take the gradients of the pixel points whose gradient magnitudes are in the top 20% to participate in the traversal, narrowing the traversal threshold range, and perform the Otsu method operation on the gradient values within this range. The specific process is as follows:

[0114] Suppose the gradient values participating in the traversal are in the interval [a, b], and there are m i windows with the gradient value of i, i ∈ [a, b], then the probability that the window with the gradient of i appears where M is the total number of windows participating in the traversal, The final ideal threshold β divides the windows into two parts: edge windows (with gradient values in the range [β, b], denoted as T windows) and non-edge windows (with gradient values in the range [a, β), denoted as F windows).

[0115] The probability p of the appearance of T windows T (denoted as the function f(β) with the threshold β as the independent variable) and the gradient mean μ T are respectively:

[0116]

[0117] The probability p of the appearance of F windows F and the gradient mean μ F are respectively:

[0118]

[0119] The total mean of the gray gradients of the image windows participating in the traversal:

[0120] μ = p T μ T + p F μ F

[0121] The between-class variance of the two types of windows:

[0122] σ 2 (β) = p T (μ T - μ) 2 + p F (μ F - μ) 2

[0123] Then the finally set threshold β * is the β that satisfies the maximum of σ 2 (β), that is * , namely

[0124]

[0125] Output the final edge detection result, screen out the windows that are not at the image edge for blind pixel detection, and thus complete the steps of image edge detection.

[0126] Next, perform blind pixel detection and confirmation. The flowchart of this part is as Figure 2 shown.

[0127] Group the pixel points in the extracted window in the horizontal direction (the area size is (2n + 1) × 1) and the vertical direction (the area size is 1 × (2n + 1)).

[0128] According to the alpha mean filtering algorithm, first remove the maximum and minimum gray values of each group of pixel points, the purpose of which is to avoid the influence of Gaussian noise and impulse noise. Calculate the average value of the remaining pixel gray values in each group, that is, the alpha filtering value H h (p) and H v (q), where H h (p) represents the filtering value of the p-th row grouping, and H v (q) represents the filtering value of the q-th column grouping.

[0129]

[0130] Among them, d is the number of removed pixels, d = 2, s represents the abscissa of any point in the q-th column of the window, and t represents the ordinate of any point in the q-th row of the window. G(p, t) is the gray value of all pixels in the p-th row within the window, and G(s, q) is the gray value of all pixels in the q-th column within the window.

[0131] Obtain the adaptive threshold required for blind pixel detection decision, which is given by a threshold model that adaptively adjusts based on temperature changes. This model specifically includes the following steps:

[0132] At the standard temperature T0, judge the blind pixel distribution by the "3σ" method. Use μ0 to represent the mean value of the gray values of the pixel points within the window at the standard temperature, and σ0 to represent the standard deviation of the gray values of the pixel points within the window at the standard temperature T0.

[0133] When the temperature T changes, the threshold α also needs to change accordingly. The present invention establishes a model for adaptively adjusting the threshold with temperature changes, and the influencing factors of the adaptive threshold are determined as: dark current I dark , signal-to-noise ratio SNR, and sensor hardware attributes.

[0134] First, the relationship between the dark current I dark and the temperature T usually satisfies the Arrhenius equation:

[0135]

[0136] By measuring multiple groups of data values before the device leaves the factory, the proportional coefficient ω of the device can be obtained, that is:

[0137]

[0138] Among them, k is the Boltzmann constant (k≈8.617×10 -5 eV / K), E α is the activation energy, taking its general value of 0.7, and ω is the proportional coefficient.

[0139] Second, the signal-to-noise ratio SNR is the ratio of the signal power to the noise power, usually expressed in decibels (dB):

[0140]

[0141] Among them, P s is the power of the image signal, and P n is the thermal noise power. When the temperature T increases, the thermal noise of the device will increase accordingly, causing the signal-to-noise ratio SNR to decrease.

[0142] Thirdly, the increase in temperature may also amplify the defects of the sensor itself, increasing the probability of the generation of blind pixels.

[0143] The present invention uses the analytic hierarchy process to give the weights of the three influences. The judgment matrix A is listed as follows:

[0144]

[0145] According to the judgment matrix A, the weight vectors W of the defects of the material itself, thermal noise, and dark current can be obtained as:

[0146] W = [0.11 0.58 0.31]

[0147] It can be seen from the final weight ratio that the influence of the defects of the material itself only accounts for 0.11, and its influence is ignored. Then the formula for the adaptive threshold α changing with temperature can be expressed as:

[0148]

[0149] Among them, is the temperature rise and fall flag, T0 = 300K, SNR is the signal-to-noise ratio measured at temperature T, SNR0 is the standard signal-to-noise ratio measured at temperature T0, and I dark is the magnitude of the dark current measured at temperature T, and I dark0 is the standard dark current magnitude measured at temperature T0.

[0150] For each pixel in the window, the absolute value of the gray value of the pixel and the alpha-filtered value of the gray values of the pixels in the corresponding row and column coordinates are obtained respectively, and used as the decision value for blind pixel detection. If the absolute value is greater than the adaptive threshold, it is set as a potential blind pixel. The judgment criteria are as follows:

[0151] |G(p,q) - H h (p)| ≥ α

[0152] |G(p,q) - H v (q)| ≥ α

[0153] Among them, G(p,q) is the gray value of any pixel in the window, and α is the adaptive threshold given by the adaptive threshold model adjusted based on temperature changes. If the absolute value does not meet the above conditions, it is not set as a potential blind pixel. Based on this, the potential blind pixel distribution matrix is constructed as follows:

[0154]

[0155] On the basis of the above judgment, for consecutive frames of images, the edges of the images are constantly changing. Therefore, there is still a certain degree of uncertainty in the above judgment. The time-domain accumulation method based on FFT pre-screening to remove periodic noise is adopted. First, periodic noise interference (such as 60Hz noise of the power supply, etc.) is excluded through frequency-domain analysis, and then high-probability blind pixels are screened through time-domain accumulation.

[0156] Assume that N potential blind pixel distribution matrices D(p, q) are obtained within N frames of images, and N can be selected according to the image processing situation. For each pixel (p, q) in the candidate blind pixel set, extract the time series G(p, q, n * ), n * ∈[0, N], n * is the frame number, that is, the n * th frame in N frames of images.

[0157] Perform FFT transformation on the time series G(p, q, n * ) to obtain F(p, q, ω), that is:

[0158]

[0159] Obtain the maximum amplitude A max (p, q):

[0160]

[0161] If the maximum amplitude A max (p, q) is greater than or equal to the frequency-domain threshold μ + 3σ, it is determined that the pixel is interfered by periodic noise; otherwise, it is considered that the pixel is not interfered by noise. Where μ and σ are the mean and standard deviation of all gray data values G(p, q, n * ) in N frames of images, n * is the frame number, that is, the n * th frame in N frames of images.

[0162] Sum up these N potential blind pixel distribution matrices to obtain the cumulative value D sum (p, q), and compare it with the blind pixel detection threshold ρD max . If it is greater than the blind pixel detection threshold, it is determined as the final blind pixel; otherwise, it is determined as a normal pixel. Where ρ is the confidence level, usually it can be taken as 0.9, and D max is the largest element in the blind pixel matrix after time-domain accumulation of multiple frames of images, that is:

[0163] D max =max{D sum(p,q)}

[0164] Let the final multi-frame image time-domain cumulative blind pixel determination matrix be DF(p,q), that is:

[0165]

[0166] Thus, the final blind pixel detection result is obtained.

[0167] The above embodiments are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An infrared blind pixel detection method based on temperature self - adaptation and time - frequency domain synthesis, characterized in that The method includes the following steps: S1. Take a (2n + 1)×(2n + 1) sliding window to traverse the infrared image, and obtain the gray values of (2n + 1) 2 pixel points around the current pixel point, denoted as G(i, j), where (i, j) is the coordinate of the central pixel. S2. Perform image edge determination. If the image within the sliding window is determined to be a non-image edge after determination, proceed to the next step; if it is determined to be an image edge, it does not participate in the dead pixel determination. S3. Group the pixel points extracted within the window in the horizontal direction (region size: (2n + 1)×1) and the vertical direction (region size: 1×(2n + 1)). S4. According to the alpha filtering algorithm, first remove the maximum and minimum values of the gray values of each group of pixel points, and then calculate the average value of the gray levels of the remaining pixels in each group, that is, the alpha filtering value H h (p) and H v (q), H h (p) represents the filtering value of the p-th row grouping, and H v (q) represents the filtering value of the q-th column grouping, S5. For each pixel within the window, calculate the absolute value of the difference between its gray value and the alpha-filtered value of the pixel group corresponding to its row coordinate and column coordinate respectively, as the decision value for dead pixel detection. If the absolute value is greater than the adaptive threshold (the adaptive threshold is given by a threshold model adaptively adjusted based on temperature changes), set it as a potential dead pixel. The judgment criteria are as follows: |G(p,q) - H h (p)| ≥ α |G(p,q) - H v (q)| ≥ α Where G(p,q) is the gray value of any pixel within the window, and α is the adaptive threshold given by the threshold model adaptively adjusted based on temperature changes. Construct a potential dead pixel distribution matrix accordingly: S6. Use the time-domain accumulation method based on FFT pre-screening to remove periodic noise interference for secondary confirmation. First, exclude periodic noise interference through frequency-domain analysis, and then screen for high-probability dead pixels through time-domain accumulation. S7. Obtain the final dead pixel detection result.

2. The infrared blind pixel detection method based on temperature self - adaptation and time - frequency domain synthesis according to claim 1, wherein, In step S1, n of the (2n + 1)×(2n + 1) sliding window takes one of the three values 1, 2, and 3.

3. The infrared blind pixel detection method based on temperature adaption and time-frequency domain synthesis according to claim 1, wherein In step S3, it specifically includes the following steps: S3.

1. When the working environment temperature of the sensor is relatively high or low, the change in sensor noise will affect the edge sharpness of the image. Give an adjustable Sobel kernel that dynamically adjusts according to the temperature range, which can dynamically adapt to different working temperature environments, reduce the impact of the change in the sensor working temperature on image edge discrimination, and ensure the accuracy of calculating the gray gradient value by the Sobel kernel. Its dynamic adjustment method is as follows: S3.

2. Obtain the gray values of each pixel of the image in sequence through a sliding window of size δ×δ. The value of δ is the same as the order of the Sobel kernel corresponding to the interval where the current working environment temperature is located. S3.

3. Normalize the gray values of the pixels within each window to the integer interval [0, 255]. This process is achieved through the following formula: Among them, G(p, q) is the gray value of a certain pixel within the window before normalization, and G * (p, q) is the gray value of the pixel at the corresponding coordinate within the window after normalization. max_Grey and min_Grey are respectively the maximum and minimum gray values of the pixels within the window. S3.

4. Perform a convolution multiplication operation on the gray values of each pixel within the window and the Sobel kernel. When δ = 5, the convolution factors in the horizontal direction and the vertical direction are respectively: Obtain the horizontal gradient G of each sampling window x and the vertical gradient G y , S3.

5. Calculate the gradient magnitude corresponding to this window. The calculation formula is as follows: Use the arithmetic mean of the absolute values instead of the square root result as follows: S3.

6. Compare the G value with the set threshold. If it is greater than the set threshold, it is determined as an image edge. The set threshold is given by the Otsu method based on gradient simplification. Arrange the gradient magnitudes of each window calculated in S3.5 from largest to smallest, and only take the gradient values of the pixel points whose gradient magnitudes are in the top 20% to participate in the traversal. Perform the Otsu method operation on the gradient values within this range. The specific process is as follows: Suppose the gradient values involved in traversal are in the interval [a, b], and there are m windows with gradient value i, where i ∈ [a, b]. i The probability that a window with gradient i appears is where M is the total number of windows involved in traversal. The final ideal threshold β divides the windows into two parts: edge windows (gradient values in the range [β, b], denoted as T windows) and non-edge windows (gradient values in the range [a, β), denoted as F windows). The probability p of the T window appearance T (denoted as the function f(β) with the threshold β as the independent variable) and the gradient mean μ T are respectively: The probability p of the F window appearing F and the gradient mean μ F are respectively Total mean value of the gray gradients of the image windows participating in the traversal: μ = p T μ T + p F μ F Between-class variance of the two types of windows: σ 2 (β) = p T (μ T - μ) 2 + p F (μ F - μ) 2 Then the final set threshold β * to satisfy σ 2 (β) is the largest β * , that is S3.7 Output the final edge detection result, and screen out the windows that are not at the image edge for dead pixel detection.

4. The infrared blind pixel detection method based on temperature self - adaptation and time - frequency domain synthesis according to claim 1, wherein: In step S4, the row / column grouped alpha filter value H h (p), H v (q) is calculated as follows: Where d is the number of pixels removed, d = 2, s represents the abscissa of any point in the q-th column of the window, t represents the ordinate of any point in the q-th row of the window, G(p, t) is the gray value of all pixels in the p-th row within the window, and G(s, q) is the gray value of all pixels in the q-th column within the window.

5. The infrared blind pixel detection method based on temperature adaption and time-frequency domain synthesis according to claim 1, wherein: In step S5, the preset threshold is given by a threshold model adaptively adjusted based on temperature change. Specifically, this model includes the following steps: At the standard temperature T0, the blind pixel distribution is judged by the "3σ" method. Let μ0 represent the mean of the gray values of the pixel points within the window at the standard temperature, and σ0 represent the standard deviation of the gray values of the pixel points within the window at the standard temperature T0. When the temperature T changes, the threshold α also needs to change accordingly. A model for adaptively adjusting the threshold with temperature change is established, and the influencing factors of the adaptive threshold are determined as: dark current I dark , signal-to-noise ratio SNR, and sensor hardware attributes First, the dark current I dark usually satisfies the Arrhenius equation with respect to the relationship with the temperature T: Before the device leaves the factory, multiple groups of data values are measured to obtain the proportionality coefficient ω of the device, that is: where k is the Boltzmann constant (k ≈ 8.617×10 -5 eV / K), E α is the activation energy, taking its general value of 0.7 eV, and ω is the proportionality coefficient, Second, the signal-to-noise ratio SNR is the ratio of the signal power to the noise power, usually expressed in decibels (dB): Among them, P s is the image signal power, and P n is the thermal noise power. When the temperature T increases, the thermal noise of the device will increase accordingly, resulting in a decrease in the signal-to-noise ratio SNR. Third, the increase in temperature may also amplify the defects of the sensor itself and increase the probability of generating blind pixels. The analytic hierarchy process is used to give the weights of the three influences, and the judgment matrix A is listed as: According to the judgment matrix A, the weight vector W of the material itself defect, thermal noise, and dark current can be obtained as: W＝[0.110.580.31] It can be seen from the final weight ratio that the influence of the material itself defect only accounts for 0.

11. Ignoring its influence, the formula for the adaptive threshold α changing with temperature is expressed as: Among them, is the temperature rise and fall flag, T0 = 300K, SNR is the signal-to-noise ratio measured at temperature T, SNR0 is the standard signal-to-noise ratio measured at temperature T0, I dark is the magnitude of the dark current measured at temperature T, I dark0 is the magnitude of the standard dark current measured at temperature T0.

6. The infrared blind pixel detection method based on temperature self - adaptation and time - frequency domain synthesis according to claim 1, characterized in that: In step S6, first, noise interference is excluded through frequency-domain analysis, and then high-probability blind pixels are screened through time-domain accumulation. The multi-condition joint decision improves the detection robustness. Specifically, it includes the following steps: S6.1 Assume that N potential blind pixel distribution matrices D(p,q) are obtained within N frames. For each pixel (p,q) in the candidate blind pixel set, extract the time series G(p,q,n * ),n * ∈[0,N], where n * is the frame number, i.e., the n * -th frame in the N frames, S6.2 Apply the FFT transform to the time series G(p, q, n * ), to obtain F(p, q, ω), that is: Obtain the maximum amplitude A of the non-zero frequency among them max (p, q): If the maximum amplitude A max (p, q) is greater than or equal to the frequency domain threshold μ + 3σ, it is determined that the pixel is interfered by periodic noise; otherwise, it is considered that the pixel is not interfered by noise, where μ and σ are the mean and standard deviation of all gray data values G(p, q, n * ) of N frames of images, and n * is the number of frames, that is, the n * -th frame in N frames of images. S6.3 Sum the N potential blind pixel distribution matrices to obtain the accumulated value D sum (p, q), and compare it with the blind pixel detection threshold ρD max If it is greater than the blind pixel detection threshold, it is determined to be the final blind pixel; otherwise, it is determined to be a normal pixel, where ρ is the confidence level with a value of 0.9, and D max is the largest element in the blind pixel matrix after the time-domain accumulation of multiple frames of images, that is: D max = max{D sum (p, q)} Denote the final multi-frame image time-domain accumulation blind pixel judgment matrix as DF(p, q), that is:

7. The infrared blind pixel detection method based on temperature self - adaptation and time - frequency domain synthesis according to claim 1, characterized in that: In step S6, the number of frames for time-domain accumulation can be flexibly selected according to the accuracy requirements of blind pixel detection and computing resources to achieve the best detection effect. The number of frames for time-domain accumulation is set to 5 - 30 frames, and according to the situation of computing resources, a frame number adaptive degradation strategy is adopted for dynamic adjustment to give priority to ensuring the blind pixel detection accuracy in the key area.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the infrared blind pixel detection method based on temperature adaption and time-frequency domain synthesis as described in any one of claims 1 to 7 above.

9. A computer-readable storage medium having computer instructions stored thereon, characterized in that, When the computer instruction is executed by the processor, it implements the infrared blind pixel detection method based on temperature adaption and time-frequency domain synthesis as described in any one of claims 1 - 7.

Images