A method and system for real-time correction of infrared image non-uniformity

Abstract

The application discloses a kind of infrared image non-uniformity real-time correction method and system, the method includes: according to the current frame image gray statistical value and the previous frame image gray statistical value output by imaging system, using the function relationship of motion angular velocity and image motion threshold value determined offline to obtain the motion angular velocity of infrared imaging system;According to the motion angular velocity of infrared imaging system, the weight of the correction parameter update calculation required for the current frame image correction is obtained, and the adaptive correction parameter update step factor is obtained according to the motion angular velocity of infrared imaging system;According to the weight of the correction parameter update calculation required for the current frame image correction and adaptive correction parameter update step factor, the correction parameter of next frame image is obtained;According to the correction parameter of next frame image, the correction of row by row and pixel by pixel of the frame image is carried out.The application solves the non-uniformity problem of short-wave infrared image detector, and ensures the real-time performance of infrared image correction algorithm.

Description

Technical Field

[0001] This invention belongs to the field of image preprocessing technology for imaging systems such as aircraft, missiles, ships, and spacecraft, and particularly relates to a method and system for real-time correction of non-uniformity in infrared images. Background Technology

[0002] In short-wave infrared imaging products, the short-wave infrared InGaAs detectors used possess advantages such as near-room temperature operation, high quantum efficiency, low dark current, and good radiation resistance, making them an ideal choice for high-sensitivity, low-power, miniaturized, and high-reliability imaging systems. However, infrared focal plane detectors, limited by material and manufacturing processes, also have their weaknesses—image non-uniformity. This problem arises fundamentally from the differences in quantum efficiency, spectral response, dark current, field of view, and temperature response among the various detector units of the infrared array detector, affecting the imaging quality and product performance. Image non-uniformity correction techniques are generally used to address this issue and improve image quality. Common methods can be divided into two categories: calibration-based image non-uniformity correction and scene-based image non-uniformity correction. Image non-uniformity correction based on calibration requires calibrating the parameters of each pixel of the infrared detector before image correction. This method is widely used, but its drawback is that the calibrated correction parameters remain unchanged. As the detector operates for longer periods, the actual parameters of the detector will drift due to changes in the working environment. If the original correction parameters are still used, the quality of the infrared image will gradually deteriorate. While scene-based image non-uniformity correction methods can solve the problem of detector parameter drift, they still have problems such as relatively complex algorithms, difficulty in hardware implementation, slow correction convergence speed, difficulty in convergence, and ghosting. Summary of the Invention

[0003] The technical problem solved by this invention is to overcome the shortcomings of the prior art and provide a method and system for real-time correction of infrared image non-uniformity, which solves the non-uniformity problem of short-wave infrared image detectors and ensures the real-time performance of infrared image correction algorithms.

[0004] The objective of this invention is achieved through the following technical solution: a method for real-time correction of non-uniformity in infrared images, comprising: obtaining the motion angular velocity of the infrared imaging system by using an offline method to determine the functional relationship between motion angular velocity and image motion threshold based on the gray-level statistical values ​​of the current frame image output by the imaging system and the gray-level statistical values ​​of the previous frame image; obtaining the weights of the calculated correction parameters required for the correction of the current frame image based on the motion angular velocity of the infrared imaging system; obtaining the step size factor of the adaptive correction parameters based on the motion angular velocity of the infrared imaging system; obtaining the correction parameters of the next frame image based on the weights of the calculated correction parameters required for the correction of the current frame image and the step size factor of the adaptive correction parameters; and correcting the next frame image row by row and pixel by pixel based on the correction parameters of the next frame image.

[0005] In the aforementioned method for real-time correction of infrared image non-uniformity, the method for offline determination of the functional relationship between motion angular velocity and image motion threshold includes: placing the infrared imaging system on a turntable, and having the infrared imaging system acquire multiple frames of original images output by the infrared detector under different angular velocities ω; obtaining the difference value in the row direction between two adjacent frames. Difference values ​​in the column direction Based on the difference in the row direction between two adjacent frames Difference values ​​in the column direction Obtain the image motion threshold; determine the functional relationship between the motion angular velocity and the image motion threshold.

[0006] In the above-mentioned method for real-time correction of infrared image non-uniformity, the difference value in the row direction of two adjacent frames is... Difference value in column direction It can be obtained through the following formula:

[0007]

[0008]

[0009]

[0010] in, This represents the difference in the row direction between two adjacent frames of the image. The difference in the column direction between two adjacent image frames, where M is the number of image columns. Output the raw grayscale value of the pixel at coordinate (i,j) in the current image (the nth frame) at temperature T to the detector. The detector outputs the raw grayscale value of the pixel at coordinate (i,j) in the previous frame (the (n-1)th frame) at temperature T. The value represents the motion detection value of the current frame image, where N is the number of image rows, j is the image row coordinate, i is the image column coordinate, n is the current frame image sequence number, and n-1 represents the previous frame image sequence number.

[0011] In the above method for real-time correction of infrared image non-uniformity, the image motion threshold is obtained by the following formula:

[0012]

[0013] Among them, thrD (ω) Let be the motion detection threshold for the image under the condition of angular velocity ω; where ω = 0, 1, 2, ..., (ω max -ω min ) / Δω, is the motion detection value of the current frame image, and m is the total number of image frames acquired under the condition of angular velocity ω.

[0014] In the above method for real-time correction of infrared image non-uniformity, the functional relationship between motion angular velocity and image motion threshold is obtained by the following formula:

[0015] ω=Z(thrD (ω) );

[0016] Where ω is the angular velocity of motion.

[0017] In the above-mentioned method for real-time correction of non-uniformity in infrared images, the weights of the correction parameters required for the correction of the current frame image are updated by using an iterative formula to construct adaptive correction parameters based on the motion angular velocity of the infrared imaging system.

[0018] In the aforementioned method for real-time correction of infrared image non-uniformity, the method for constructing the iterative formula for adaptive correction parameters includes: calculating the error between the corrected image and the desired image. Based on the error between the corrected image and the desired image The correction parameters are obtained through a real-time update iterative formula using the most shaky descent method.

[0019] In the above-mentioned method for real-time correction of infrared image non-uniformity, the error between the corrected image and the desired image... It can be obtained through the following formula:

[0020]

[0021]

[0022]

[0023] in, To correct the error between the grayscale value and the desired grayscale value for the pixel at coordinate (i,j) in the nth frame image, Let be the corrected grayscale value of the (i,j)th pixel in the nth frame image. Correct the desired grayscale value for the pixel at coordinate (i,j) in the nth frame image. Let be the gain correction coefficient for the pixel at coordinate (i,j) in the nth frame image. The detector outputs the grayscale value for the pixel at coordinate (i,j) in the nth frame image. The pixel offset correction coefficient for the nth frame image coordinate (i,j) is given. The detector outputs a grayscale value for the pixel at coordinates (i-1, j-1) in the nth frame of the image. The detector outputs a grayscale value for the pixel at coordinate (i,j-1) in the nth frame image. The detector outputs the grayscale value for the pixel at coordinates (i+1, j-1) in the nth frame image. The detector outputs the grayscale value for the pixel at coordinate (i-1,j) in the nth frame image. The detector outputs the grayscale value for the pixel at coordinate (i,j+1) in the nth frame image. The detector outputs the grayscale value for the pixel at coordinates (i+1, j-1) in the nth frame image. The detector outputs the grayscale value for the pixel at coordinate (i+1,j) in the nth frame image. The detector outputs the grayscale value for the pixel at coordinates (i+1, j+1) in the nth frame image, where T is the current temperature value, j is the row coordinate of the image, i is the column coordinate of the image, and n is the frame number.

[0024] In the above method for real-time correction of infrared image non-uniformity, when the image frame sequence n is not greater than the preset value p, the real-time update iterative formula for the correction parameters is obtained through the following formula:

[0025]

[0026]

[0027] in, Let be the gain correction coefficient for the pixel at coordinate (i,j) in the (n+1)th frame. Let γ be the gain correction coefficient for the pixel at coordinate (i,j) in the nth frame, and γ be the correction step size factor. The detector outputs the grayscale value for the pixel at coordinate (i,j) in the nth frame image. To correct the error between the grayscale value and the desired grayscale value for the pixel at coordinate (i,j) in the nth frame image, The offset correction coefficient for pixel coordinates (i,j) in the (n+1)th frame image. , where T is the pixel offset correction coefficient at coordinate (i,j) of the nth frame image, j is the row coordinate of the image, i is the column coordinate of the image, and n is the frame number;

[0028] When the image frame sequence n is greater than the preset value p, the real-time update formula for the correction parameters is obtained through the following formula:

[0029]

[0030]

[0031] in, Let be the gain correction coefficient for the pixel at coordinate (i,j) in the (n+1)th frame. Let k1 be the pixel offset correction coefficient at coordinate (i,j) of the (n+1)th frame image, k2 be the weight of the correction coefficient of the nth frame image in the iterative calculation of the correction coefficient of the (n+1)th frame image, k3 be the weight of the correction coefficient of the (n-2)th frame image in the iterative calculation of the correction coefficient of the (n-3)th frame image in the iterative calculation of the correction coefficient of the (n+1)th frame image, k4 be the weight of the correction coefficient of the (n-4)th frame image in the iterative calculation of the correction coefficient of the (n+1)th frame image, and k5 be the weight of the correction coefficient of the (n-4)th frame image in the iterative calculation of the correction coefficient of the (n+1)th frame image. p The weights of the image correction coefficients for the (n-p+1)th frame in the iterative calculation of the image correction coefficients for the (n+1)th frame. Let be the gain correction coefficient for the pixel at coordinate (i,j) in the nth frame image. Let be the gain correction coefficient for the pixel at coordinate (i,j) in the (n-1)th frame of the image. Let be the gain correction coefficient for the pixel at coordinate (i,j) in the (n-2)th frame. Let be the gain correction coefficient for pixel coordinate (i,j) in the (n-3)th frame. Let be the gain correction coefficient for pixel coordinate (i,j) in the (n-4)th frame. Let be the gain correction coefficient for pixel coordinate (i,j) in the (n-p+1)th frame of the image. The pixel offset correction coefficient for the nth frame image coordinate (i,j) is given. The pixel offset correction coefficient for the (i,j) coordinates in the (n-1)th frame image. The pixel offset correction coefficient for the (i,j) coordinates in the (n-2)th frame image. The pixel offset correction coefficient for the (i,j) coordinates in the (n-3)th frame image. The pixel offset correction coefficient for the (i,j) coordinates in the (n-4)th frame image. It is the pixel offset correction coefficient for the (i,j) coordinates of the image in the (n-p+1)th frame.

[0032] In the above-mentioned method for real-time correction of non-uniformity in infrared images, the adaptive correction parameter update step size factor is obtained by calculating the functional relationship between the adaptive correction parameter update step size factor and the angular velocity of the infrared imaging system offline.

[0033] A system for real-time correction of non-uniformity in infrared images includes: a first module for obtaining the motion angular velocity of the infrared imaging system by using an offline method to determine the functional relationship between motion angular velocity and image motion threshold based on the gray-level statistical values ​​of the current frame image output by the imaging system and the gray-level statistical values ​​of the previous frame image; a second module for updating the calculated weights of the correction parameters required for the correction of the current frame image based on the motion angular velocity of the infrared imaging system, and updating the step size factor based on the adaptive correction parameters; a third module for obtaining the correction parameters of the next frame image based on the updated calculated weights and the updated step size factor of the adaptive correction parameters; and a fourth module for correcting the next frame image row by row and pixel by pixel based on the correction parameters of the next frame image.

[0034] Compared with the prior art, the present invention has the following advantages:

[0035] (1) In this invention, the correction parameters of the first frame of the adaptive parameter update are based on pre-calibrated initial correction parameters, rather than being corrected from the original image. This allows the correction algorithm to converge quickly and calculate the correction parameters of the next frame in real time. When the image frame sequence is greater than a certain number of frames p, the correction parameters of the current p frames are used to iteratively calculate the correction parameters of the next image frame.

[0036] (2) The present invention parametrically designs the step size factor, which can be adjusted in real time in the register according to the correction effect, and collects the images output by the infrared detector under different working conditions in advance, and selects the appropriate step size factor according to the different application environment and the characteristics of the detection target.

[0037] (3) The present invention can correct each original pixel of the infrared image in real time when it arrives, which can maximize the real-time processing requirements, reduce the processing time, and finally the real-time corrected image frame rate can be synchronized to the detector exposure frame rate. Attached Figure Description

[0038] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0039] Figure 1 This is a flowchart illustrating the real-time correction method for non-uniformity of infrared detectors according to the present invention.

[0040] Figure 2 This is a schematic diagram of the hardware structure for real-time correction of infrared image non-uniformity according to the present invention.

[0041] Figure 3This is a schematic diagram illustrating the calculation of the desired image;

[0042] Figure 4 A schematic diagram illustrating the iterative update calculation for the correction parameters;

[0043] Figure 5 This is a schematic diagram showing the storage after iterative updates of the correction parameters. Detailed Implementation

[0044] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0045] Figure 1 This is a flowchart illustrating the real-time correction method for non-uniformity of infrared detectors according to the present invention. Figure 1 The method for real-time correction of infrared image non-uniformity is characterized by comprising:

[0046] Based on the grayscale statistical values ​​of the current frame image output by the imaging system and the grayscale statistical values ​​of the previous frame image, the motion angular velocity of the infrared imaging system is obtained by using an offline method to determine the functional relationship between motion angular velocity and image motion threshold.

[0047] The weights for updating the correction parameters required for the current frame image correction are obtained by constructing an iterative formula for the adaptive correction parameters based on the motion angular velocity of the infrared imaging system. The adaptive correction parameter update step size factor is obtained by calculating the functional relationship between the adaptive correction parameter update step size factor and the angular velocity offline based on the motion angular velocity of the infrared imaging system.

[0048] The calculated weights k1 to k are updated based on the correction parameters required for the current frame image correction. p The correction parameters for the next frame image are obtained by updating the step size factor γ using (0,1) and adaptive correction parameters.

[0049] Based on the correction parameters of the next frame image The next frame of the image is corrected row by row and pixel by pixel.

[0050] Specifically, the steps of this method are as follows:

[0051] (1) Under uniform light radiation conditions at different temperatures T, the non-uniformity correction parameters (gain parameters and bias parameters) of the infrared image detector are calibrated. The specific steps are as follows:

[0052] (1-1) Based on the correction model of each detection unit of a classic infrared detector:

[0053] V i,j(φ) =R i,j ×DN i,j(φ) +O i,j

[0054] The detector resolution is M×N, where V i,j(φ) R is the grayscale value output by the i,j (i∈[1,M],j∈[1,N]) detector unit in the array after non-uniformity correction calculation under the light radiation intensity condition φ. i,j Let O be the correction gain parameter for the i,j-th detector unit. i,j DN is the correction bias parameter for the i,j-th detector element. i,j(φ) Let φ be the original image value output by the detector under the condition of light radiation intensity φ of the i-th and j-th detector units in the array.

[0055] (1-2) Adjust the light radiation intensity of the light source output to φ1 and φ2 respectively (φ1>φ2, the detector outputs a grayscale value of 80% of the saturation value under light radiation intensity of φ1, and a grayscale value of 20% of the saturation value under light radiation intensity of φ2). Collect 100 frames of output images of the infrared detector under these two light radiation conditions respectively, and calculate the time-domain mean of each detection unit in these 100 frames. According to the model of each detection unit, its response output is as follows:

[0056]

[0057] Where DN i,j(φ1) DN i,j(φ2) These are the original grayscale values ​​of the detector's i and j (i∈[1,M], j∈[1,N]) detector units under light radiation intensity conditions φ1 and φ2, respectively.

[0058] (1-3) Take the average pixel space Under uniform light incidence conditions, the ideal case for correction of the entire image plane is as follows: Therefore:

[0059]

[0060] The calibrated gain and bias parameters can be calculated as follows:

[0061]

[0062] (1-4) Adjust the ambient temperature. The selected temperature range should be consistent with the actual temperature range of the detector being used. T∈(T min ,T max ), from T min Initially, temperature data is collected at intervals of ΔT (where ΔT = 5℃), with the last temperature point being T. max Repeat steps (1-1) to (1-3) to calculate the gain parameter matrix R at different ambient temperature points T. i,j(T) With bias parameter matrix O i,j(T) The initial values ​​of the parameter matrix for the correction of the first frame of infrared image are obtained. and

[0063] (2) Initialize the infrared correction parameter matrix from step (1). and The data is converted into a binary file and burned into the onboard Nand Flash memory chip as the initial value for subsequent adaptive parameter correction calculations.

[0064] (3) Determine the functional relationship between image motion angular velocity and image motion monitoring threshold offline: ω=Z(thrD) (ω) ).

[0065] (3-1) Place the imaging system on a carrier platform such as a turntable that can output a fixed angular velocity, and set the rotational speed of the carrier to ω∈(ω min ,ω max (Take ω) min =0deg / s, ω max =20deg / s), step size Δω (take Δω = 1deg / s), acquire m frames of raw images output by the infrared detector under different angular velocity ω rotation conditions of the infrared imaging system (take m = 500).

[0066] (3-2) Under the condition of angular velocity input of ω rotation, calculate the total mean of row and column vectors for the nth frame and the (n-1)th frame of the input image sequence, and calculate the difference between them to obtain the difference value of row direction between adjacent frames. Difference value in column direction As shown below:

[0067]

[0068]

[0069]

[0070] in, This represents the difference in the row direction between two adjacent frames of the image. The difference in the column direction between two adjacent frames is M, where M is the number of image columns. Output the raw grayscale value of the pixel at coordinates (i,j) in the current image (the nth frame) at temperature T. The detector outputs the raw grayscale value of the pixel at coordinate (i,j) in the previous frame (the (n-1)th frame) at temperature T. The value represents the motion detection value of the current frame image, where N is the number of image rows, j is the image row coordinate, i is the image column coordinate, n is the current frame image sequence number, and n-1 represents the previous frame image sequence number.

[0071] (3-3) Take Among them, thrD (ω) Let be the motion detection threshold for the image under the condition of angular velocity ω. Where ω = 0, 1, 2, ..., (ω... max -ω min ) / Δω, in deg / s. This represents the motion detection value for the current frame of the image.

[0072] (3-4) Using [0,1,…,ω,…,20] and [thrD0,thrD1,…,thrD] ω ,…,thrD 20 The angular velocity ω with respect to the motion detection threshold thrD is calculated using the least squares method. (ω) The relationship.

[0073] (3-5)ω=Z(thrD (ω) The relationship is a continuous function. In this embodiment, to facilitate hardware implementation and reduce computational load, it is discretized into three segments (image at rest, image moving at low speed, and image moving at high speed). The angular velocities of each segment are represented by ω0, ω1, and ω2, respectively. min ≤ω0≤ω1≤ω2≤ω max The image motion angular velocity thresholds are represented by thr0 and thr1 respectively (where thr0 = thrD). (1) thr0 = thrD (10) ).

[0074] when At that time, the image is considered to be in a stationary state with an angular velocity of ω0.

[0075] when At that time, the image is considered to be in a low-speed motion state with an angular velocity of ω1.

[0076] when At that time, the image is considered to be in a state of high-speed motion with an angular velocity of ω2.

[0077] (4) Using the idea of ​​adaptive parameters based on minimum mean square error, the iterative formula of adaptive correction parameters is constructed by the most shaky descent method.

[0078] (4-1) The non-uniformity correction model adopted is the classic two-point correction model. For the nth frame of the image output by the detector (the current frame in the image frame sequence), the correction result output by the i,j (i∈[1,M],j∈[1,N]) detector unit in the detector is... As shown below:

[0079]

[0080] in Let be the correction gain parameter for the i,j-th detector unit in the n-th frame image. Let be the correction bias parameters for the i,j-th detector unit in the n-th frame image. is the original image grayscale value of the i,j detection unit in the nth frame (current frame). T is the current temperature value.

[0081] (4-2) In order to obtain a desired image that is closer to the ideal image, it is necessary to calculate a corrected image desired value for each pixel of the image after correction in the current frame. In this scheme, the expected value is taken as the average of the 8 neighboring pixels in the space:

[0082]

[0083] in Let be the expected grayscale value of the i,j-th detector unit in the n-th frame of the image. The original image grayscale value of the i,j detection unit in the nth frame. Let be the 8 raw pixels surrounding the i-th and j-th detection units in the n-th frame image. Let T be the current temperature value.

[0084] (4-3) Since there will definitely be a deviation between the actual corrected image and the expected image, it is necessary to calculate the corrected image. With the desired image error

[0085]

[0086] The core idea of ​​adaptive correction based on minimum mean square error is that the corrected image is approximately equal to or similar to the image expected to be estimated. Minimum, and based on the independent response of each detection unit of the infrared focal plane detector, that is, it is necessary to Minimum. For the correction parameter R i,j and O i,jFind the partial derivatives. Construct iterative correction formulas for R and O using the most erratic descent method, and obtain the basic correction parameters to update the iterative formulas in real time.

[0087]

[0088]

[0089] Wherein, γ is the step size factor for image non-uniformity correction, which is used to adjust the convergence speed of image correction and is parameterized according to the application of the algorithm. Let be the gain correction coefficient for the pixel at coordinate (i,j) in the (n+1)th frame. is the pixel offset correction coefficient for the (i,j) coordinates of the (n+1)th frame image. and These are the initial image correction parameters obtained from the calibration in the first step, used for the correction of the first frame image. T is the current temperature value.

[0090] (4-4) During the correction process, correction parameters for four consecutive frames are cached (default p = 4) for subsequent optimal parameter calculation. When the current image frame number n < 4, the iterative update formula for the correction parameters of the next frame uses the basic correction parameter update iterative formula described in (4-3). Starting from the nth frame (n ≥ 4), the correction parameters are selected from the previous four frames to iteratively calculate the optimal parameters for the correction of the next frame, as follows:

[0091]

[0092]

[0093] in, Let be the gain correction coefficient for the pixel at coordinate (i,j) in the (n+1)th frame. Let be the pixel offset correction coefficient at coordinate (i,j) in the (n+1)th frame. k1~k4∈(0,1) are the weighting coefficients selected for the correction parameters, and k1+k2+k3+k4=1. The selection of these weighting coefficient values ​​depends on the motion state of the image. Ideally, when the image is stationary, the inter-frame correction parameters do not change significantly, and the ideal correction parameters are close to the average value. However, when the image is in motion, the inter-frame correction parameters change significantly, and the ideal correction parameters should be closer to the most recently calculated correction parameters.

[0094] When the image is determined to be stationary, with an angular velocity of ω0, the values ​​of the adjustment parameters k1~k4∈(0,1) are:

[0095] k1=0.25, k2=0.25, k3=0.25, k4=0.25

[0096] When the image is determined to be in a relatively low-speed motion state with an angular velocity of ω1, the adjusted values ​​of the correction parameter weights k1~k4∈(0,1) are:

[0097] k1=0.5, k2=0.3125, k3=0.125, k4=0.0625

[0098] When the image is determined to be in a state of relatively high-speed motion, with an angular velocity of ω2, the adjusted values ​​of the correction parameter weights k1~k4∈(0,1) are:

[0099] k1 = 1, k2 = 0, k3 = 0, k4 = 0

[0100] By substituting the calculated image correction parameters for the next frame into the correction formula in (4-1), the image correction for the next frame (the (n+1)th frame) can be performed.

[0101] (5) Based on the actual application environment and the characteristics of the detected target, acquire multiple frames of original images output by the infrared detector under different working conditions, such as stationary targets and moving targets with different angular velocities. Based on the acquired original images and the initial values ​​of the correction parameters calculated in step (2), and The formula for iterative parameter calculation in step (4-3) is used to calculate the appropriate step size factor γ offline.

[0102] (5-1) Acquire 500 frames of raw images from the infrared detector under static conditions of the target and background. Combine the 500-frame image sequence of the raw static target with the initial values ​​of the correction parameters. and Substituting into the formula for parameter iteration in (4-3), we set the step size factor γ∈(1×e -7 ,1×e -6 Let Δγ = 1 × e -7 Simulation correction calculations were performed on the original image. The smoothness ρ of the corrected image after 500 frames was statistically analyzed when using various step size factors γ.

[0103]

[0104] In the formula, DN is the gray value matrix of the image to be evaluated, v1 = [1, -1] is the horizontal gradient operator, and v2 = [1, -1] T The vertical gradient operator is denoted by *, where * represents matrix convolution and || ||1 represents finding the first normal form of the matrix.

[0105] Calculate the mean flatness of 500 image frames

[0106] Take a step size factor γ1 to make The minimum value is used as the step size factor for adjusting the optimal correction parameter when the image is stationary (the angular velocity of the carrier is ω0).

[0107] (5-2) Place the imaging system on a carrier platform such as a turntable that can output a fixed angular velocity. Set the carrier to rotate at a low speed of 5 degrees / s and a high speed of 15 degrees / s respectively. Acquire 500 frames of raw images output by the infrared detector under the motion conditions of the target and the background. Combine the 500-frame image sequence of the raw moving target with the initial values ​​of the correction parameters. and Substituting into the formula for parameter iteration in (4-3), we set the step size factor γ∈(1×e -6 ,1×e -5 Let Δγ = 1 × e -6 Simulation correction calculations are performed on the original images. The signal-to-noise ratio (PSNR) of the corrected images is calculated statistically using various step size factors γ for 500 frames.

[0108]

[0109] In the formula DN i,j F represents the grayscale value of the image to be evaluated. i,j Let be the expected value of the image, the image size is M×N, b is the number of bits of the gray value of a single pixel in the image, j is the row coordinate of the image, and i is the column coordinate of the image.

[0110] Calculate the mean signal-to-noise ratio of 500 frames of images.

[0111] Taking a step size factor γ to make the image at a low speed of 5deg / s The maximum value is used as the step size factor γ2 for adjusting the optimal image correction parameters under low-speed motion conditions (carrier motion angular velocity is ω1).

[0112] Taking a step size factor γ to make the image at a high speed of 15 deg / s The maximum value is used as the step size factor γ3 for adjusting the optimal correction parameter of the image under high-speed motion (carrier motion angular velocity is ω2).

[0113] (6) The hardware implementation block diagram of the entire infrared detector non-uniformity correction is as follows: Figure 2 As shown, when the infrared imaging system starts working, it first caches the initial values ​​of the calibration parameters obtained from calibration under different temperature fields pre-stored in Flash memory into the DDR memory. Among these, the gain parameters for the same temperature field... With bias parameters They are stored in the same address space so that the hardware can read them within one pixel clock cycle, ensuring real-time calculation. The infrared imaging system measures the temperature of the infrared detector in real time and selects the calibration parameters at the corresponding temperature as the initial calibration values ​​before each frame of image exposure.

[0114] (7) The FPGA of the infrared imaging system caches the raw image data output by the infrared detector through exposure control and raw data acquisition. The image pixels of 2 rows are cached in the internal register of the FPGA for image correction calculation and subsequent calculation.

[0115] (8) When the infrared detector outputs the first frame of the original image From the first row and first column of pixels When transmitting data line by line and pixel by pixel sequentially, it is passed through the multiplier module and the adder module in pipeline order. The input parameter of the multiplier module is the initial gain parameter of the calibration. The input parameters of the adder module are the initial bias parameters of the scaling. Obtain the non-uniformity correction output of the first frame image.

[0116] (9) The calculation of the desired image is accomplished by caching the original data of two rows of the image. The specific hardware implementation method is as follows: Figure 3 As shown in the figure, when the original image is transmitted sequentially to the (i+1)th, (j+1)th pixel unit, the eight adjacent pixels at coordinates i,j have already been buffered. Calculating their average value yields the expected image value of the i,jth image unit.

[0117] (10) such as Figure 2 The non-uniformity correction output obtained in step (8) is shown. Compared with the desired image value obtained in step (9) Simultaneously, the error between the corrected image and the desired image is obtained by feeding the data into a subtractor. And compare it with the correction parameters of the current frame. and Original image Simultaneously, the data is sent to the calibration parameter update module.

[0118] (11) The correction parameter update module calculates the gain parameter of the next frame image based on the formula for real-time iteration of the correction parameters obtained in step (4). With bias parameters A schematic diagram illustrating the specific calibration parameter iteration hardware update is shown below. Figure 4 As shown.

[0119] (12) After the new correction parameters for the next frame are calculated, they are cached in the DDR through the MIG interface inside the FPGA. In each iteration, the correction parameters of the new frame image are calculated to replace the correction parameters of the same pixel coordinates of the previous frame image in the DDR, that is ( cover cover A diagram illustrating the cache is shown below. Figure 5 As shown.

[0120] (13) In this way, after each frame of image is corrected, the detector temperature is read in real time and image motion detection calculation is performed to select the appropriate correction step size factor and correction parameters for the next frame of image and iteratively update the weights until the end.

[0121] This embodiment also provides a system for real-time correction of infrared image non-uniformity, comprising: a first module, used to obtain the motion angular velocity of the infrared imaging system by using a method of offline determination of the functional relationship between motion angular velocity and image motion threshold based on the gray-level statistical values ​​of the current frame image output by the imaging system and the gray-level statistical values ​​of the previous frame image; a second module, used to update the calculated weights of the correction parameters required for the correction of the current frame image based on the motion angular velocity of the infrared imaging system, and to update the step size factor of the adaptive correction parameters based on the motion angular velocity of the infrared imaging system; a third module, used to obtain the correction parameters of the next frame image by updating the calculated weights and the step size factor of the adaptive correction parameters based on the correction parameters required for the correction of the current frame image; and a fourth module, used to correct the next frame image row by row and pixel by pixel based on the correction parameters of the next frame image.

[0122] This embodiment, based on the traditional infrared two-point correction method, adds adaptive calculation of the minimum mean square error parameter, tracks and updates the non-uniformity correction parameters of the infrared image in real time, detects the detector temperature in real time, and selects appropriate initial values ​​of correction parameters within different temperature ranges. It does not require the use of mechanical structures such as shutters, thus overcoming the problem of random drift of detector correction parameters with working time and temperature changes in the traditional two-point correction method.

[0123] In this embodiment, the correction parameters for the first frame of the adaptively updated image are based on pre-calibrated initial correction parameters, rather than being corrected from the original image. This allows the correction algorithm to converge quickly and calculate the correction parameters for the next frame in real-time closed-loop tracking. When the image frame sequence exceeds a certain number of frames p, the correction parameters for the next image frame are iteratively calculated using the correction parameters of the current p frames.

[0124] This embodiment uses parameterized design for the step size factor, which can be adjusted in real time in the register according to the correction effect. Images output by the infrared detector under different working conditions are acquired in advance, and the appropriate step size factor is selected according to the different application environments and the characteristics of the detection targets.

[0125] The multi-threaded pipelined hardware structure of this embodiment implements the main correction calculation process inside the FPGA. It can correct each original pixel of the infrared image in real time as it arrives, which can maximize the real-time processing requirements, reduce processing time, and finally synchronize the real-time corrected image frame rate to the detector exposure frame rate.

[0126] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solutions of the present invention by utilizing the methods and techniques disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the protection scope of the technical solutions of the present invention.

Claims

1. A method for real-time correction of non-uniformity in infrared images, characterized in that... include: Based on the grayscale statistical values ​​of the current frame image output by the imaging system and the grayscale statistical values ​​of the previous frame image, the motion angular velocity of the infrared imaging system is obtained by using an offline method to determine the functional relationship between motion angular velocity and image motion threshold. The weights of the calculation are updated based on the correction parameters required for the current frame image correction obtained from the motion angular velocity of the infrared imaging system, and the step size factor is updated based on the adaptive correction parameters obtained from the motion angular velocity of the infrared imaging system. The calculated weights and adaptive correction parameters are updated based on the correction parameters required for the current frame image correction to update the step size factor and obtain the correction parameters for the next frame image. The next frame image is corrected row by row and pixel by pixel based on the correction parameters of the next frame image; Methods for determining the functional relationship between motion angular velocity and image motion threshold offline include: The infrared imaging system is placed on a turntable, and the infrared imaging system acquires multiple frames of raw images output by the infrared detector under different angular velocity conditions; Obtain the difference values ​​in the row direction and column direction of two adjacent image frames; The image motion threshold is obtained by calculating the difference in row direction and the difference in column direction between two adjacent frames. Determine the functional relationship between motion angular velocity and image motion threshold; The differences in the row and column directions between two adjacent image frames are obtained using the following formulas: in, This represents the difference in the row direction between two adjacent frames of the image. The difference in the column direction between two adjacent image frames, where M is the number of image columns. The detector outputs the original grayscale value of the pixel at coordinate (i,j) in the nth frame image at temperature T. The detector outputs the original grayscale value of the pixel at coordinate (i,j) in the (n-1)th frame image at temperature T. The current frame image motion detection value is N, where N is the number of rows in the image, j is the row coordinate of the image, i is the column coordinate of the image, n is the frame number of the current frame image, and n-1 represents the frame number of the previous frame image. The image motion threshold is obtained using the following formula: Among them, thrD (ω) Let ω be the motion detection threshold for the image under the condition of angular velocity ω. The value represents the motion detection value of the current frame image, and m represents the total number of image frames acquired under the condition of angular velocity ω. The functional relationship between motion angular velocity and image motion threshold is obtained through the following formula: ω=Z(thrD (ω) ); Where ω is the angular velocity of motion.

2. The method for real-time correction of infrared image non-uniformity according to claim 1, characterized in that: The weights for updating the correction parameters required for the current frame image correction are obtained by constructing an iterative formula for adaptive correction parameters based on the motion angular velocity of the infrared imaging system.

3. The method for real-time correction of infrared image non-uniformity according to claim 2, characterized in that: Methods for constructing iterative formulas for adaptive correction parameters include: Calculate the error between the corrected image and the desired image; The correction parameters are updated in real time using an iterative formula based on the steepest descent method, which is used to obtain the correction parameters according to the error between the corrected image and the desired image.

4. The method for real-time correction of infrared image non-uniformity according to claim 3, characterized in that: The error between the corrected image and the desired image is obtained by the following formula: in, To correct the error between the grayscale value and the desired grayscale value for the pixel at coordinate (i,j) in the nth frame image, Let be the corrected grayscale value of the (i,j)th pixel in the nth frame image. Correct the desired grayscale value for the pixel at coordinate (i,j) in the nth frame image. Let be the gain correction coefficient for the pixel at coordinate (i,j) in the nth frame image. The detector outputs the grayscale value for the pixel at coordinate (i,j) in the nth frame image. The pixel offset correction coefficient for the nth frame image coordinate (i,j) is given. The detector outputs the grayscale value for the pixel at coordinates (i-1, j-1) in the nth frame image. The detector outputs the grayscale value for the pixel at coordinate (i,j-1) in the nth frame image. The detector outputs the grayscale value for the pixel at coordinates (i+1, j-1) in the nth frame image. The detector outputs the grayscale value for the pixel at coordinate (i-1,j) in the nth frame image. The detector outputs the grayscale value for the pixel at coordinate (i,j+1) in the nth frame image. The detector outputs the grayscale value for the pixel at coordinates (i+1, j-1) in the nth frame image. The detector outputs the grayscale value for the pixel at coordinate (i+1,j) in the nth frame image. The detector outputs the grayscale value for the pixel at coordinates (i+1, j+1) in the nth frame image, where T is the current temperature value, j is the row coordinate of the image, i is the column coordinate of the image, and n is the frame number.

5. The method for real-time correction of infrared image non-uniformity according to claim 3, characterized in that: When the frame number n is not greater than the preset value p, the real-time update formula for the correction parameters is obtained through the following formula: in, Let be the gain correction coefficient for the pixel at coordinate (i,j) in the (n+1)th frame. Let γ be the gain correction coefficient for the pixel at coordinate (i,j) in the nth frame, and γ be the correction step size factor. The detector outputs the grayscale value for the pixel at coordinate (i,j) in the nth frame image. To correct the error between the grayscale value and the desired grayscale value for the pixel at coordinate (i,j) in the nth frame image, The offset correction coefficient for pixel coordinates (i,j) in the (n+1)th frame image. , where T is the pixel offset correction coefficient at coordinate (i,j) of the nth frame image, j is the row coordinate of the image, i is the column coordinate of the image, and n is the frame number; When the frame number n is greater than the preset value p, the real-time update formula for the correction parameters is obtained through the following formula: in, Let be the gain correction coefficient for the pixel at coordinate (i,j) in the (n+1)th frame. Let k1 be the pixel offset correction coefficient at coordinate (i,j) of the (n+1)th frame image, k2 be the weight of the correction coefficient of the nth frame image in the iterative calculation of the correction coefficient of the (n+1)th frame image, k3 be the weight of the correction coefficient of the (n-2)th frame image in the iterative calculation of the correction coefficient of the (n-3)th frame image in the iterative calculation of the correction coefficient of the (n+1)th frame image, k4 be the weight of the correction coefficient of the (n-4)th frame image in the iterative calculation of the correction coefficient of the (n+1)th frame image, and k5 be the weight of the correction coefficient of the (n-4)th frame image in the iterative calculation of the correction coefficient of the (n+1)th frame image. p This represents the weight of the image correction coefficients for the (n-p+1)th frame in the iterative calculation of the image correction coefficients for the (n+1)th frame. Let be the gain correction coefficient for the pixel at coordinate (i,j) in the nth frame image. Let be the gain correction coefficient for the pixel at coordinate (i,j) in the (n-1)th frame of the image. Let be the gain correction coefficient for the pixel at coordinate (i,j) in the (n-2)th frame. Let be the gain correction coefficient for pixel coordinate (i,j) in the (n-3)th frame. Let be the gain correction coefficient for pixel coordinate (i,j) in the (n-4)th frame. Let be the gain correction coefficient for pixel coordinate (i,j) in the (n-p+1)th frame of the image. The pixel offset correction coefficient for the nth frame image coordinate (i,j) is given. The pixel offset correction coefficient for the (i,j) coordinates in the (n-1)th frame image. The pixel offset correction coefficient for the (i,j) coordinates in the (n-2)th frame image. The pixel offset correction coefficient for the (i,j) coordinates in the (n-3)th frame image. The pixel offset correction coefficient for the (i,j) coordinates in the (n-4)th frame image. It is the pixel offset correction coefficient for the (i,j) coordinates of the image in the (n-p+1)th frame.

6. A system for real-time correction of non-uniformity in infrared images, characterized in that... include: The first module is used to obtain the motion angular velocity of the infrared imaging system by using an offline method to determine the functional relationship between motion angular velocity and image motion threshold based on the gray-scale statistical values ​​of the current frame image output by the imaging system and the gray-scale statistical values ​​of the previous frame image. The second module is used to update the calculated weights based on the correction parameters required for the current frame image correction obtained from the motion angular velocity of the infrared imaging system, and to update the step size factor based on the adaptive correction parameters obtained from the motion angular velocity of the infrared imaging system. The third module is used to update the calculated weights and adaptive correction parameters based on the correction parameters required for the current frame image correction to update the step size factor and obtain the correction parameters for the next frame image. The fourth module is used to correct the next frame image line by line and pixel by pixel based on the correction parameters of the next frame image; Methods for determining the functional relationship between motion angular velocity and image motion threshold offline include: The infrared imaging system is placed on a turntable, and the infrared imaging system acquires multiple frames of raw images output by the infrared detector under different angular velocity conditions; Obtain the difference values ​​in the row direction and column direction of two adjacent image frames; The image motion threshold is obtained by calculating the difference in row direction and the difference in column direction between two adjacent frames. Determine the functional relationship between motion angular velocity and image motion threshold; The differences in the row and column directions between two adjacent image frames are obtained using the following formulas: in, This represents the difference in the row direction between two adjacent frames of the image. The difference in the column direction between two adjacent image frames, where M is the number of image columns. The detector outputs the original grayscale value of the pixel at coordinate (i,j) in the nth frame image at temperature T. The detector outputs the original grayscale value of the pixel at coordinate (i,j) in the (n-1)th frame image at temperature T. The current frame image motion detection value is N, where N is the number of image rows, j is the image row coordinate, i is the image column coordinate, n is the current frame image frame number, and n-1 represents the previous frame image frame number. The image motion threshold is obtained using the following formula: Among them, thrD (ω) Let ω be the motion detection threshold for the image under the condition of angular velocity ω. The value represents the motion detection value of the current frame image, and m represents the total number of image frames acquired under the condition of angular velocity ω. The functional relationship between motion angular velocity and image motion threshold is obtained through the following formula: ω=Z(thrD (ω) ); Where ω is the angular velocity of motion.

Images