A high frame rate super large dynamic range implementation method under a single scene

Abstract

This invention relates to a method for achieving high frame rate and ultra-large dynamic range in a single scenario. The method involves determining the radiation characteristics of the target based on the detection task and obtaining the spectral range; determining the detector type based on the obtained spectral range and detection requirements; and determining the integration time increments based on the target's radiation characteristics, the dynamic range requirements of the detection scenario, and the capabilities of the detector devices, and calculating the integration time for each increment. T int_N The invention involves calculating the absolute radiometric calibration coefficients for each time level; performing multi-time integration imaging of the detected scene using a remote sensor; and real-time blind pixel replacement and radiometric correction processing on the integrated time imaging results for each time level using a video processing device. A saturation threshold is set, and the relationship between the image DN value and the saturation threshold is determined pixel-by-pixel. The DN value assignment operation is then performed iteratively to generate a fused image without saturation and rich in detail. This invention achieves ultra-large dynamic range target detection without increasing image data capacity.

Description

Technical Field

[0001] This invention belongs to the field of aerospace optical remote sensor technology, and relates to a method for realizing high frame rate and ultra-large dynamic range in a single scene. Background Technology

[0002] With the rapid development of aerospace optical remote sensing technology, the requirements for the dynamic range of optical remote sensors are also increasing. The theoretical maximum dynamic range of a 12-bit image from a traditional infrared optical remote sensor is 4096:1. A single integration time is insufficient to display the detailed levels of a real scene's large dynamic range in a single image. This will prevent target detection and recognition from achieving the expected results. In order to meet the requirements of a large dynamic range, it is necessary to design a high frame rate, large dynamic range, and multi-level exposure adaptive image fusion scheme for the detector to achieve ultra-large dynamic range target detection. Summary of the Invention

[0003] The technical problem solved by this invention is to overcome the shortcomings of the prior art and propose a method for realizing high frame rate and ultra-large dynamic range in a single scene, so as to achieve ultra-large dynamic range target detection without increasing the image data capacity.

[0004] The solution to the technical problem of this invention is: a method for achieving high frame rate and ultra-large dynamic range in a single scene, comprising the following steps: Determine the radiation characteristics of the target and obtain its spectral range based on the detection mission; Determine the detector type based on the acquired spectral range and detection requirements; Based on the target's radiation characteristics, the dynamic range requirements of the detection scenario, and the capabilities of the detector devices, determine the integration time increments and calculate the integration time required for each increment. T int_N Then, for the integration time of each level, absolute radiometric calibration data are collected, and the absolute radiometric calibration coefficient of each level is calculated. The optical remote sensor performs multi-level integration time and high frame rate imaging of the detected scene. The video processing equipment of the remote sensor performs real-time blind pixel replacement processing on the integration time imaging results of each level, and performs radiometric correction processing based on the absolute radiometric calibration coefficient of each level. Set a saturation threshold, determine the relationship between the image DN value and the saturation threshold at each integration time after blind pixel replacement and radiometric correction, and perform DN value assignment operations traversally to generate a fused image without saturation and rich in detail.

[0005] Furthermore, determining the radiation characteristics of the target includes target type, radiation range, and spectral range; The methods of representing radiation range include: DR= 20 log10 ( L_max / L_min ) in, DR This refers to the dynamic range of radiation intensity. L_max This represents the highest radiance in the scene. L_min The lowest radiance in the scene; The spectral range is determined by the strongest radiation band of the target in different wavelengths of visible light, near infrared, mid-infrared, or long-infrared.

[0006] Furthermore, the method of representing the radiation range also includes: DR = L_max / L_min .

[0007] Furthermore, the conditions for determining the detector type include: The detector's frame rate is no less than 4Hz, the array size is no less than 2k×2k, and the long integration time is no less than 80ms.

[0008] Furthermore, the method for determining the integration time level is as follows:

[0009] in, N For points time tiers, To round up, DN bit The range of DN values ​​corresponding to the number of quantization bits of the detector.

[0010] Furthermore, the integral time required to calculate each gear position... T int_N The method is as follows: allocate the upper and lower energy ranges for dynamic imaging of the remote sensor at this level, and then calculate the required integration time based on the upper and lower energy ranges, the infrared detector's integrating capacitance, and the quantum efficiency parameter. T int_N .

[0011] Furthermore, when performing two-level integration time imaging, the traversal execution of DN value assignment operation includes: When fusing long and short integration time images, values ​​are assigned pixel by pixel: if the DN value of the current pixel in the long integration time exposure mode is less than the saturation threshold, the output DN value is the image DN value in the long integration time exposure mode, and the fusion flag is 0, indicating that the fused image is long integration time image data. If the DN value of the current pixel in the long integration time exposure mode is greater than or equal to the saturation threshold, the output DN value is the image DN value in the short integration time exposure mode, and the fusion flag is 1, indicating that the fused image data is short integration time image data.

[0012] Furthermore, when performing three-level integration time imaging, the traversal execution of DN value assignment operation includes: When fusing images with long, medium, and short integration times, values ​​are assigned pixel by pixel: if the DN value of the current pixel in the long integration time exposure mode is less than the saturation threshold, the output DN value is the DN value of the image in the long integration time exposure mode. If the DN value of the current pixel in the long integration time exposure mode is equal to the saturation threshold, then determine the DN value of the medium integration time exposure mode; if the DN value of the current pixel in the medium integration time exposure mode is less than the saturation threshold, then output the DN value as the DN value of the medium integration time exposure mode, or output the DN value as the DN value of the short integration time exposure mode. If the DN value of the current pixel in the medium integration time exposure mode is equal to the saturation threshold, then the output DN value is the DN value of the image in the short integration time exposure mode.

[0013] A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the steps of the method for implementing a high frame rate and ultra-large dynamic range in a single scenario are disclosed.

[0014] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method for achieving a high frame rate and ultra-large dynamic range in a single scenario.

[0015] The advantages of this invention compared to the prior art are: (1) The method for realizing high frame rate and ultra-large dynamic range in a single scene of the present invention solves the problem of insufficient dynamic range of traditional target detection, and has positive significance for the development of ultra-large dynamic range target detection of optical remote sensors; (2) This invention targets ultra-large dynamic response scenarios. It automatically performs multi-level integration time exposure process through optical remote sensor to collect images with different integration times. The video processor automatically calculates and fuses the image data to generate new remote sensing images, thereby improving the dynamic response range of the detector. This method also reduces background noise and improves detection sensitivity. Attached Figure Description

[0016] Figure 1 A flowchart illustrating a method for implementing high frame rate and ultra-large dynamic range in a single scenario; Figure 2 This is a flowchart of the long and short integration time cyclic exposure process in an embodiment of the present invention; Figure 3 This is a flowchart of the three-level integral time cyclic exposure process in an embodiment of the present invention; Figure 4The atmospheric transmittance at different altitudes and in multiple spectral bands is shown in the embodiments of the present invention. Detailed Implementation

[0017] like Figure 1 As shown, the present invention proposes a method for achieving high frame rate and ultra-large dynamic range in a single scene, comprising the following steps: S1. Determine the radiation characteristics of the target and obtain the spectral range based on the detection mission.

[0018] The radiation characteristics of the target are determined, including the target type, radiation range, and spectral range. The target type refers to the object being detected using the method of this invention, such as natural scenery, high-temperature objects like engine exhaust, or low-temperature objects like nocturnal organisms.

[0019] Based on the target type, further clarify the dynamic range of the target's strongest and weakest radiation intensities. The dynamic range can be expressed by the following formula: DR= 20 log 10 ( L_max / L_min ) in, DR For dynamic range, L_max This represents the highest radiance in the scene. L_min This represents the lowest radiance in the scene. Alternatively, it can be expressed directly as the ratio of the highest to the lowest radiance, as shown in the following formula: DR = L_max / L_min The target's strongest radiation bands in different wavelengths, such as visible light, near-infrared, mid-infrared, or long-infrared, are determined to obtain the spectral range.

[0020] S2. Determine the detector type based on the spectral range and detection requirements obtained in step S1.

[0021] Different types of detectors (such as MCT, CCD, CMOS, InGaAs) have their specific response bands, dynamic range substrates, and noise characteristics.

[0022] The detector selected in this invention must possess high frame rate imaging capability. To meet the energy accumulation requirements of high frame rate imaging, the detector selected in this invention must also be a large-area array, providing a long integration time and the ability to perceive images with a large dynamic range. For infrared detection, an MCT detector is preferred, requiring a frame rate of no less than 4Hz, an array size of no less than 2k×2k, and a long integration time of no less than 80ms.

[0023] S3. Based on the target's radiation characteristics, the dynamic range requirements of the detection scenario, and the capabilities of the detector devices themselves, determine the integration time increments and calculate the integration time required for each increment. Tint_N Then, for the integration time of each gear, absolute radiation calibration data is collected, and the absolute radiation calibration coefficient of each gear is calculated.

[0024] In step S3, the method for determining the integration time level is as follows:

[0025] N For points time tiers, DN bit This refers to the range of DN values ​​corresponding to the number of bits quantized by the detector. For example, the value corresponding to 14-bit quantization is 16384. This is for rounding up.

[0026] The integral time required to calculate each gear level T int_N The method is as follows: allocate the upper and lower energy ranges for dynamic imaging of the remote sensor at this setting, and then calculate the required integration time based on parameters such as the upper and lower energy ranges, the infrared detector integrating capacitance, and quantum efficiency. T int_N .

[0027] S4. The optical remote sensor performs multi-level integration time and high frame rate imaging of the detected scene. The video processing equipment of the remote sensor performs real-time blind pixel replacement processing on the integration time imaging results of each level, and performs radiometric correction processing based on the absolute radiometric calibration coefficient of each level.

[0028] S5. Set a saturation threshold, determine the relationship between the image DN value and the saturation threshold at each integration time after blind pixel replacement and radiometric correction, and perform DN value assignment operations traversally to generate a fused image with no saturation and rich details.

[0029] In step S5, the default algorithm prioritizes imaging image information with a longer integration time, but the threshold can be adjusted based on the level of attention given to the image information. Taking two integration times as an example, long and short integration times generate two 12-bit images, which are then merged by a video processor to output a single 12-bit image. During image synthesis, each pixel is assigned a value using a 2-out-of-1 method. The assignment principle is as follows: if the DN value of the current pixel in long exposure mode is less than 4095 (saturation threshold, adjustable), the output DN value is the DN value of the image in long exposure mode; if the DN value of the current pixel in long exposure mode is 4095 (saturation threshold, adjustable), the output DN value is the DN value of the image in short exposure mode.

[0030] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0031] Example (1) Determine the radiation characteristics of the target based on the satellite detection mission; In this embodiment, the target type to be detected is a target with a certain temperature, the temperature dynamic range requirement is 200000:1, the maximum value of the digital signal output by the camera is 0~4095, and its maximum response value is substituted into the absolute radiometric calibration equation to obtain the maximum detectable radiance, which is used as the upper limit of the camera's detection capability; the DN value of the background radiation is substituted into the calibration equation as the minimum detectable target to obtain the radiance of the minimum detectable target, which is used as the lower limit of the camera's detection capability.

[0032] The camera can further and significantly extend its dynamic range by setting multiple integration times for alternating exposures. The camera employs a two-time integration time alternating exposure mode, achieving a ratio of 200,000:1, which meets the needs of large dynamic range observation of the target.

[0033] The spectrum band detected in the examples is that of a space exploration system, according to Figure 4 The atmospheric transmittance of different optical system spectral bands is given in the figure. The spectral band with better transmittance at the target altitude is selected. In this embodiment, the short and mid-wave infrared spectral band is selected.

[0034] (2) Determine the detection spectrum and detector type; Assume the detector is a 2048×2048 area array with multiple taps, a readout rate of 10MHz, and a frame rate of 8Hz.

[0035] (3) Continuous detection is carried out by using high frame rate and multi-level integral time cyclic exposure; Set multiple sets of exposure parameters for the target being observed.

[0036] a. In some embodiments, in a single scene, long and short integration time exposures are automatically performed to generate two images of the detected target. The video processor performs image fusion processing on the two adjacent frames of short integration time image data and long integration time image data into one frame image, which is output at a frame rate of 8Hz. Figure 2 As shown.

[0037] During long and short integration time image fusion, values ​​are assigned pixel by pixel. If the DN value of the current pixel in the long integration time exposure mode is less than the threshold (90% of the saturated DN value, which can be set), the output DN value is the image DN value in the long integration time exposure mode, and the fusion flag is 0. If the image DN value of the current pixel in the long integration time exposure mode is greater than or equal to 90% of the saturated DN value, the output DN value is the image DN value in the short integration time exposure mode, and the fusion flag is 1.

[0038] Image fusion should synchronously add a fusion identifier to each pixel's image data. "1" indicates that the fused image data is short integration time image data, and "0" indicates that the fused image data is long integration time image data. The fusion image identifier of all pixels is placed in the highest bit of the quantized pixel image data to form 15 bits of image data, where the 15th bit is the fusion image data identifier bit.

[0039] b. In other embodiments, in a single scene, three levels of integration time exposure are performed to generate three images of the detected target information. The video processor performs image fusion processing on the three adjacent frames of short integration time image data, medium integration time image data, and long integration time image data into one frame of image data for output. For example... Figure 3 As shown.

[0040] When fusing images with three integration time settings, a three-choice method is used to assign values ​​to each pixel. If the DN value of the current pixel in the long integration time exposure mode is less than the threshold (4095, which can be set), the output DN value is the DN value of the image in the long exposure mode. If the DN value of the current pixel in the long integration time exposure mode is equal to the threshold 4095, then determine the DN value of the medium integration time exposure mode; if the DN value of the current pixel in the medium integration time exposure mode is less than the threshold 4095, then output the DN value as the DN value of the medium integration time exposure mode, or output the DN value as the DN value of the short integration time exposure mode. If the DN value of the current pixel in the medium integration time exposure mode is equal to the threshold 4095, then the output DN value is the DN value of the image in the short integration time exposure mode.

[0041] (4) Large dynamic range image output.

[0042] Image fusion enables target detection with an ultra-wide dynamic range. Long integration times enable stable and effective detection of faint targets, while short integration times enable unsaturated detection of targets with radiation intensities up to millions of W / sr.

[0043] In summary, the method of this invention achieves target detection with high frame rate and ultra-large dynamic range in a single scene.

[0044] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solutions of the present invention by utilizing the methods and techniques disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the protection scope of the technical solutions of the present invention.

[0045] The contents not described in detail in this specification are common knowledge to those skilled in the art.

Claims

1. A method for realizing high frame rate and ultra dynamic range in a single scene, characterized in that, Includes the following steps: Determine the radiation characteristics of the target and obtain its spectral range based on the detection mission; Determine the detector type based on the acquired spectral range and detection requirements; Based on the target's radiation characteristics, the dynamic range requirements of the detection scenario, and the capabilities of the detector devices, determine the integration time increments and calculate the integration time required for each increment. T int_N Then, for the integration time of each level, absolute radiometric calibration data are collected, and the absolute radiometric calibration coefficient of each level is calculated. The optical remote sensor performs multi-level integration time and high frame rate imaging of the detected scene. The video processing equipment of the remote sensor performs real-time blind pixel replacement processing on the integration time imaging results of each level, and performs radiometric correction processing based on the absolute radiometric calibration coefficient of each level. Set a saturation threshold, determine the relationship between the image DN value and the saturation threshold at each integration time after blind pixel replacement and radiometric correction, and perform DN value assignment operations traversally to generate a fused image without saturation and rich in detail.

2. The method for achieving high frame rate and ultra-large dynamic range in a single scene according to claim 1, characterized in that, The determination of the radiation characteristics of the target includes target type, radiation range, and spectral range; The methods of representing radiation range include: DR= 20 log 10 ( L_max / L_min ) in, DR This refers to the dynamic range of radiation intensity. L_max This represents the highest radiance in the scene. L_min The lowest radiance in the scene; The spectral range is determined by the strongest radiation band of the target in different wavelengths of visible light, near infrared, mid-infrared, or long-infrared.

3. The method for achieving high frame rate and ultra-large dynamic range in a single scene according to claim 2, characterized in that, The representation of the radiation range also includes: DR = L_max / L_min 。 4. The method for achieving high frame rate and ultra-large dynamic range in a single scene according to claim 1, characterized in that, The conditions for determining the detector type include: The detector's frame rate is no less than 4Hz, the array size is no less than 2k×2k, and the long integration time is no less than 80ms.

5. The method for achieving high frame rate and ultra-large dynamic range in a single scene according to claim 2, characterized in that, The method for determining the integration time level is as follows: in, N For points time tiers, To round up, DN bit The range of DN values ​​corresponding to the number of quantization bits for the detector.

6. The method for achieving high frame rate and ultra-large dynamic range in a single scene according to claim 1, characterized in that, The integral time required to calculate each gear level T int_N The method is as follows: allocate the upper and lower energy ranges for dynamic imaging of the remote sensor at this level, and then calculate the required integration time based on the upper and lower energy ranges, the infrared detector's integrating capacitance, and the quantum efficiency parameter. T int_N .

7. The method for achieving high frame rate and ultra-large dynamic range in a single scene according to claim 1, characterized in that, When performing two-level integration time imaging, the traversal execution of DN value assignment operation includes: When fusing long and short integration time images, values ​​are assigned pixel by pixel: if the DN value of the current pixel in the long integration time exposure mode is less than the saturation threshold, the output DN value is the image DN value in the long integration time exposure mode, and the fusion flag is 0, indicating that the fused image is long integration time image data. If the DN value of the current pixel in the long integration time exposure mode is greater than or equal to the saturation threshold, the output DN value is the image DN value in the short integration time exposure mode, and the fusion flag is 1, indicating that the fused image data is short integration time image data.

8. The method for achieving high frame rate and ultra-large dynamic range in a single scene according to claim 1, characterized in that, When performing three-level integration time imaging, the traversal execution of DN value assignment operation includes: When fusing images with long, medium, and short integration times, values ​​are assigned pixel by pixel: if the DN value of the current pixel in the long integration time exposure mode is less than the saturation threshold, the output DN value is the DN value of the image in the long integration time exposure mode. If the DN value of the current pixel in the long integration time exposure mode is equal to the saturation threshold, then determine the DN value of the medium integration time exposure mode; if the DN value of the current pixel in the medium integration time exposure mode is less than the saturation threshold, then output the DN value as the DN value of the medium integration time exposure mode, or output the DN value as the DN value of the short integration time exposure mode. If the DN value of the current pixel in the medium integration time exposure mode is equal to the saturation threshold, then the output DN value is the DN value of the image in the short integration time exposure mode.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 8.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 8.

Images