Imaging dynamic range expansion method based on variable integral time

Abstract

The invention discloses an imaging dynamic range expansion method based on variable integral time, and relates to the field of infrared imaging. The method comprises the following steps of: firstly, sending an infrared original image acquired by a medium-wave infrared movement to an FPGA (Field Programmable Gate Array) chip, decoding integral time of the infrared original image, and converting a signal format; then the integral time information and the converted infrared image signal are synchronously sent to a non-uniformity correction module, and non-uniformity noise is corrected; performing dynamic range expansion of the fused image signal on the corrected image signal through a variable integral time imaging algorithm; and finally, digitally displaying the obtained fused image signal. The core innovation of the method is that a hierarchical integral time segmentation calibration model is established, and a non-uniformity correction parameter dynamically matched with integral time is obtained. The method breaks through the physical limitation of single-frame integration time, the low-temperature background signal-to-noise ratio is remarkably improved while high-temperature target details are reserved, and the method is particularly suitable for the refined imaging requirement of a high-dynamic complex infrared scene.

Description

Technical Field

[0001] The invention belongs to the field of infrared focal plane array imaging and relates to an imaging dynamic range expansion method based on variable integration time. Background Art

[0002] According to the blackbody radiation law, all objects in nature with a temperature above absolute zero will continuously generate infrared thermal radiation. The radiation energy is proportional to the fourth power of the surface temperature of the object, and the main energy is concentrated in the mid- and far-infrared bands (3-14μm). In real-world applications, the temperature difference between the target and the environment can reach several orders of magnitude: for example, the tail flame temperature of a high-speed aircraft can exceed 2000K, while the background temperature of the atmosphere in which it is located may be less than 200K. This extreme temperature gradient poses a severe challenge to infrared imaging systems: when imaging with a fixed integration time, the high-temperature target area will produce saturation distortion due to radiation energy overload, while the low-temperature background area will obliterate details due to insufficient signal strength, forming the inherent contradiction of "high temperature cutoff and low temperature saturation".

[0003] The existing imaging dynamic range expansion technology has the following bottlenecks:

[0004] (1) Insufficient dynamic range expansion. Traditional methods achieve dynamic range expansion through inter-frame accumulation, but their theoretical limit is limited by the detector potential well capacity, and the measured dynamic range is usually less than 100dB. In extremely high-dynamic scenarios (temperature gradient > 1000K) such as missile explosions and target combustion, weak targets will still be overwhelmed by the "halo" effect formed by strong radiation sources, making it difficult to meet the requirements of precision guidance and battlefield situation awareness.

[0005] (2) Time domain fusion produces motion artifacts. In medium-wave infrared (3-5μm) applications, it is often necessary to extend the integration time to the millisecond level to improve the signal-to-noise ratio of low-temperature backgrounds. When a high-speed moving target (speed > 100m / s) shifts between frames, the accumulation of multiple frames will produce a significant "tailing" effect, and the target energy will be dispersed along the motion trajectory, resulting in a decrease in the equivalent signal-to-noise ratio, which seriously affects the effectiveness of the target recognition algorithm.

[0006] (3) Non-uniformity correction parameter mismatch. Existing technologies often use non-uniformity correction parameters with fixed integration time. When the integration time changes dynamically, the detector response characteristics and the preset correction parameters will mismatch, resulting in fixed pattern noise in the fused image, which is particularly obvious in low-temperature background areas.

[0007] Therefore, an imaging dynamic range expansion method based on variable integration time is needed to solve the above problems. Summary of the Invention

[0008] Purpose of the invention: The purpose of the present invention is to provide an imaging dynamic range expansion method based on variable integration time, so as to achieve dynamic range expansion and improve image quality.

[0009] Technical solution: To achieve the above-mentioned purpose, the present invention provides a method for expanding the dynamic range of imaging based on variable integration time, comprising the following steps:

[0010] Step 1: Collect original infrared images;

[0011] Step 2: Decode the integration time information of the infrared original image to obtain the infrared original images at different integration times, and convert the infrared original images into a standard format;

[0012] Step 3: Obtain the non-uniformity correction gain correction parameters and offset correction parameters of the infrared original image at different integration times, and use the correction parameters to process the non-uniform noise and random noise contained in the infrared original image to obtain the infrared corrected image;

[0013] Step 4: Using the variable integration time fusion algorithm, the infrared correction images at different integration times are fused into a single-frame high dynamic infrared image;

[0014] Step 5: Display the fused infrared image as a digital image.

[0015] Among them, the medium-wave infrared core is used to collect infrared original images.

[0016] When collecting the original infrared image, the integration time information is embedded into the image to record the integration time of the frame image.

[0017] Among them, the integral time satisfies Tint1<Tint2<……<Tint n The step distribution of Tint n Indicates the nth integration time.

[0018] Among them, the field programmable gate array FPGA chip is used to decode the integral time information of the infrared original image and convert the image format.

[0019] The format conversion is to convert a two-dimensional image into one-dimensional data for processing.

[0020] Among them, the method for processing the noise contained in the infrared original image using the non-uniformity correction gain correction parameter is as follows: first, the non-uniformity correction gain correction parameters of the infrared original image under different integration times are obtained by the non-uniformity correction method. Specifically, when collecting the infrared original image, two temperature values ​​are set, which are lower than the low-temperature cutoff threshold of the detector and higher than the high-temperature saturation threshold of the detector. The two-point method in the non-uniformity correction is used to calculate the non-uniformity correction gain correction parameters under the integration time based on the detector response images under the two temperature values. Then, by setting the integration time, the non-uniformity correction gain parameters corresponding to each integration time under the variable integration time are obtained. Then, based on the gain correction parameters, gain correction is performed on the non-uniform noise and random noise of the infrared image.

[0021] Among them, the method for processing the noise contained in the infrared original image using the non-uniformity correction offset correction parameter is: setting a variable integration time to obtain the non-uniformity correction offset correction parameter under different integration times, and using the offset correction parameter to perform offset correction on the non-uniform noise and random noise in the image based on the image after gain correction.

[0022] Among them, the method of using the variable integration time fusion algorithm to fuse infrared corrected images under different integration times into a single-frame high-dynamic infrared image is as follows: quantizing the grayscale level of each sub-frame infrared corrected image, and generating a reference grayscale space based on the grayscale level, based on the multi-integration time image, using the pixel-level grayscale value superposition and fusion method to obtain a cumulative image with doubled grayscale levels, linearly mapping the grayscale of the obtained cumulative image to the original reference grayscale space to obtain a single-frame high-dynamic infrared image.

[0023] The dynamic range D of the single-frame high dynamic image is r The calculation method is:

[0024]

[0025] Among them, T n Preset high temperature point, V' satn is the average gray value of the image, V min1 is the minimum grayscale value in the image, V nRMS For noise reading, iterates over all pixels and returns the image standard deviation as the image noise reading.

[0026] Beneficial effects: The present invention uses multi-integration time subframe fusion technology to significantly improve the signal-to-noise ratio of the low-temperature background while retaining the details of the high-temperature target, thereby achieving dynamic range expansion and breaking through the physical limitations of the single-frame integration time; the variable integration time fusion output meets the imaging requirements of high-speed moving targets; improves image quality and effectively eliminates motion artifacts and fixed pattern noise; it is suitable for highly dynamic and complex scenes such as missile explosions and target combustion, and can simultaneously enhance the texture details of high-temperature targets and low-temperature backgrounds. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is the flow chart of the method;

[0028] Figure 2 The comparison diagrams before and after the grayscale direct accumulation method are shown. Figure (a) shows a single-frame image without the grayscale direct accumulation method, and Figure (b) shows a multi-frame accumulated image after the grayscale direct accumulation method.

[0029] Figure 3 These are high-dynamic scene images at different integration times, where (a) is an image with an integration time of 10μs, (b) is an image with an integration time of 83μs, (c) is an image with an integration time of 216μs, and (d) is an image obtained with eight different integration times. DETAILED DESCRIPTION

[0030] The present invention is further described in detail below with reference to the accompanying drawings.

[0031] The present invention discloses a method for expanding the imaging dynamic range based on a variable integration time. The principle is as follows: Using a variable integration time image acquisition method, multiple sub-frame images are output within the original output time of a single frame. The integration time used for each sub-frame image is arranged to maximize the response range of the infrared focal plane pixel. Furthermore, a variable integration non-uniformity correction method is employed to individually correct the non-uniformity of the infrared focal plane at different integration times, ensuring that the resulting sub-frame images all have good non-uniformity. Finally, using image accumulation, the sub-frame images are combined into a single frame, expanding the image dynamic range and changing the frame rate to achieve normal image output. Different scenes require different variable integration time ranges. As the integration time increases, the output gradually increases, and the target becomes clearer. However, when the integration time is too long, saturation occurs. As the integration time decreases, the output gradually decreases, and the target becomes dim, potentially leading to cutoff. Therefore, selecting an appropriate integration time is crucial. In the present invention, adjusting the variable integration time range can achieve better imaging results.

[0032] According to the characteristics of infrared detectors, the original infrared image has inevitable non-uniformity and random noise. The non-uniformity correction of the original infrared image is performed by obtaining the non-uniformity correction gain correction parameters and the offset correction parameters. The image after noise correction is directly accumulated and fused by the variable integral time image fusion algorithm.

[0033] The fusion process is as follows: first, the grayscale level of the infrared correction image is quantized, and a reference grayscale space is generated according to this quantization level. Then, based on the variable integral time image, the pixel-level grayscale value is directly superimposed and fused to obtain a cumulative image with doubled grayscale levels. Finally, since the cumulative image cannot be displayed after the grayscale level is doubled, the grayscale of the accumulated image is linearly mapped to the original reference grayscale space to obtain a single-frame high-dynamic infrared image with a multi-level display effect of different scene objects.

[0034] like Figure 1 、 2 As shown, according to the above principle, a method for expanding the imaging dynamic range based on variable integration time of the present invention includes the following steps:

[0035] Step 1: The medium-wave infrared core outputs the original image at a frame rate of 100±1Hz, and makes the integration time of each single frame different, cycles at a fixed period, and records the integration time information at the same time.

[0036] By analyzing the detector signal and timing diagram, we can know that the detector detects the integral time signal T int When the rising edge is reached, the output of the new sub-frame image begins. It is necessary to make the integration time of each sub-frame image different so as to obtain a variable integration time image. Set n different integration times as T int1 、T int2 ,……,T intn When setting the integration time, it is important to note that the blackbody integral temperature is suitable for short integration times. For long integration times, the output of the detector pixel may have reached saturation, making it impossible to effectively read out the data and, naturally, to effectively calibrate the coefficients. In this case, the blackbody temperature used in both calibrations must be lowered to keep the pixel output value within a reasonable range.

[0037] Step 2: After the original infrared image is sent to the FPGA, the integration time information is decoded and the image information is converted to a standard format for subsequent algorithm processing. Using a four-channel detector output, the detector outputs a subframe image in one-quarter the original time. The four simultaneous outputs are then combined into a single channel for subsequent transmission and processing.

[0038] Step 3: Process the severe non-uniformity and random noise contained in the original infrared image, and calculate the non-uniformity correction coefficients of the original infrared images obtained at different integration times.

[0039] Based on the response model of the imaging system, the present invention proposes an adaptive integration time correction method to meet the correction requirements of rapid changes.

[0040] The adaptive integration time correction method consists of two parts: non-uniformity correction gain correction and offset correction. The method for obtaining the non-uniformity correction gain parameters for different integration times is to first set two temperatures within the low-temperature cutoff and high-temperature saturation temperature thresholds to ensure that the pixel detector response avoids low-temperature cutoff and high-temperature saturation. Then, based on the detector response images at the two temperature values, the non-uniformity correction gain correction parameters for the integration time are calculated using the "two-point method" in the non-uniformity correction method. Then, by setting the integration time, the non-uniformity correction gain parameters corresponding to each integration time are obtained. Finally, the gain-corrected infrared image is obtained based on the gain correction parameters.

[0041] The method for obtaining the offset correction parameters for non-uniformity correction under different integration times is relatively easy and does not require calibration and acquisition in advance. It can be obtained and corrected in real time during program execution. The method is as follows: set the current integration time, control the FPGA to drop the baffle, take the average of multiple sub-frame images to obtain the offset correction parameters, and put them into the corresponding address in DDR3 for future use. Set the variable integration time to obtain the offset correction parameters for non-uniformity correction under variable integration time. Finally, based on the gain-corrected image, the offset correction parameters are used to obtain the offset-corrected infrared image, i.e., the infrared correction image.

[0042] Step 4: With the help of the variable integral time fusion algorithm, multiple sub-frames contained in the same group of images are fused into a single-frame high-dynamic infrared image.

[0043] The variable integral time fusion algorithm adopts the grayscale direct accumulation method. The method is as follows: first, the grayscale level of the infrared correction image is quantized, and a reference grayscale space is generated according to this quantization level. Then, based on multiple frames of variable integral time images, the pixel-level grayscale value is directly superimposed and fused to obtain an accumulated image with doubled grayscale levels. Finally, since the accumulated image cannot be displayed after the grayscale level is doubled, the grayscale of the accumulated image is linearly mapped to the original reference grayscale space to obtain a single-frame high dynamic infrared image.

[0044] The dynamic range of a single-frame high dynamic image is calculated by the relevant readings, and the dynamic range D r The calculation formula is:

[0045]

[0046] Among them, T n Preset high temperature point, V' satn is the average gray value of the read image, V min1 is the minimum grayscale value in the image, obtained from the grayscale value reading at -30℃, V nRMS For noise reading, iterates over all pixels and returns the image standard deviation as the image noise reading.

[0047] Step 5: Display the fused infrared image as a digital image.

[0048] like Figure 3 As shown in the figure, taking the medium-wave infrared core outputting the original image at a frame rate of 100±1Hz, and selecting eight different integration times: 4μs, 10μs, 17μs, 34μs, 83μs, 133μs, 167μs and 216μs to implement variable integration time image fusion as an example, the specific implementation process of the present invention is as follows:

[0049] The medium-wave infrared core outputs raw images at a frame rate of 100±1Hz, with an image size of 640*512 and a detection spectrum of 3μm-5μm. The medium-wave infrared core collects images in a fixed cycle according to the eight pre-set integration times and records the integration time information;

[0050] After the original infrared image is sent to the FPGA, the integration time information is decoded, and the four simultaneously output data are converted into one data for subsequent transmission and processing; then, dynamic blind pixel correction and adaptive integration time non-uniformity correction are used to remove the noise of the original image, and the dynamic blind pixel detection scheme and judgment method are used to remove the dynamic blind pixel. The non-uniformity correction parameters are calculated through the non-uniformity correction gain correction parameter acquisition method and the offset correction parameter acquisition method to correct the random noise and non-uniform noise of the original image.

[0051] The corrected images are subjected to variable-integral time image fusion using the direct grayscale accumulation method. First, a reference grayscale space model is established to generate a non-uniformity-corrected reference grayscale space, and the grayscale levels are quantized. Dynamic range expansion is then achieved through a linear accumulation operation, with the quantization levels satisfying a multiplication relationship. In this example, the reference grayscale space quantization level is 16384, and the multiplication relationship is the same as the number of subframes being fused, which is 8.

[0052] Combine Figure 2 Figure (a) shows a single-frame image without the grayscale direct accumulation method, and Figure (b) shows a multi-frame cumulative image after the grayscale direct accumulation method. The imaging dynamic range expansion method based on variable integration time of the present invention has a good effect on improving the image signal-to-noise ratio.

[0053] Combine Figure 3 , where (a) is the image with an integration time of 10μs, (b) is the image with an integration time of 83μs, (c) is the image with an integration time of 216μs, and (d) is the image obtained with eight different integration times. Figure 3 It can be seen that the fused image can present the details of high-temperature objects in the scene at low integration time, and can also capture low-temperature objects at high integration time. The overall image has clear layers, rich details, and a high dynamic range.

[0054] The present invention provides a method for expanding the imaging dynamic range based on a variable integration time. This method can enhance the imaging dynamic range and improve the image signal-to-noise ratio (SNR) to a certain extent. This method requires selecting an appropriate variable integration time range based on the specific scenario to achieve the optimal dynamic range expansion effect. This method overcomes the physical limitations of a single-frame integration time, significantly improving the low-temperature background SNR while preserving high-temperature target details. It is particularly suitable for the refined imaging needs of highly dynamic and complex infrared scenes.

Claims

1. A method for expanding the dynamic range of imaging based on variable integration time, characterized by: The following steps are involved: Step 1: Collect original infrared images; Step 2: Decode the integration time information of the infrared original image to obtain the infrared original images at different integration times, and convert the infrared original images into a standard format; Step 3: Obtain the non-uniformity correction gain correction parameters and offset correction parameters of the infrared original image at different integration times, and use the correction parameters to process the non-uniform noise and random noise contained in the infrared original image to obtain the infrared corrected image; Step 4: Using the variable integration time fusion algorithm, the infrared correction images at different integration times are fused into a single-frame high dynamic infrared image; Step 5: Display the fused infrared image as a digital image.

2. The imaging dynamic range expansion method based on variable integration time according to claim 1, characterized in that: The medium-wave infrared core is used to collect infrared raw images.

3. The imaging dynamic range expansion method based on variable integration time according to claim 1, characterized in that: When collecting infrared original images, the integration time information is embedded into the image to record the integration time of the frame image.

4. The imaging dynamic range expansion method based on variable integration time according to claim 3, characterized in that: The integral time satisfies Tint1<Tint2<……<Tint n The step distribution of Tint n Indicates the nth integration time.

5. The imaging dynamic range expansion method based on variable integration time according to claim 1, characterized in that: The field programmable gate array (FPGA) chip is used to decode the integral time information of the original infrared image and convert the image format.

6. The imaging dynamic range expansion method based on variable integration time according to claim 5, characterized in that: The format conversion is to convert a two-dimensional image into one-dimensional data for processing.

7. The imaging dynamic range expansion method based on variable integration time according to claim 1, characterized in that: A method for processing the noise contained in the infrared original image using the non-uniformity correction gain correction parameter is as follows: first, the non-uniformity correction gain correction parameters of the infrared original image under different integration times are obtained by the non-uniformity correction method. Specifically, when collecting the infrared original image, two temperature values ​​are set, which are lower than the low-temperature cutoff threshold of the detector and higher than the high-temperature saturation threshold of the detector. The non-uniformity correction gain correction parameters under the integration time are calculated based on the detector response images under the two temperature values ​​using the two-point method in the non-uniformity correction. Then, by setting the integration time, the non-uniformity correction gain parameters corresponding to each integration time under the variable integration time are obtained. Finally, based on the gain correction parameters, gain correction is performed on the non-uniform noise and random noise of the infrared image.

8. The imaging dynamic range expansion method based on variable integration time according to claim 1, characterized in that: The method for processing the noise contained in the infrared original image using the non-uniformity correction offset correction parameters is as follows: setting a variable integration time to obtain the non-uniformity correction offset correction parameters under different integration times, and using the offset correction parameters to perform offset correction on the non-uniform noise and random noise in the image based on the gain corrected image.

9. The imaging dynamic range expansion method based on variable integration time according to claim 1, characterized in that: The method of using a variable integration time fusion algorithm to fuse infrared corrected images at different integration times into a single-frame high-dynamic infrared image is as follows: quantizing the grayscale level of each sub-frame infrared corrected image, generating a reference grayscale space based on the grayscale level, and using a pixel-level grayscale value superposition and fusion method based on multiple integration time images to obtain a cumulative image with doubled grayscale levels. The grayscale of the obtained cumulative image is linearly mapped to the original reference grayscale space to obtain a single-frame high-dynamic infrared image.

10. The imaging dynamic range expansion method based on variable integration time according to claim 9, characterized in that: The dynamic range D of the single-frame high dynamic image r The calculation method is: Among them, T n Preset high temperature point, V' satn is the average gray value of the image, V min1 is the minimum grayscale value in the image, V nRMS For noise reading, iterates over all pixels and returns the image standard deviation as the image noise reading.

Images