Infrared image acquisition method and device with adjustable integration time

Abstract

The present invention discloses an infrared image acquisition method and device with adjustable integration time, including: configuring an initial integration time based on a working environment, and collecting infrared images under the initial integration time; using a photoelectric detection system to track a target based on the collected infrared image; in the process of tracking the target: determining the image complexity of a local image where the target exists according to the collected infrared image; and determining the signal-to-noise ratio of the local image where the target exists; adjusting the integration time of the infrared detector of the photoelectric detection system based on the image complexity and the signal-to-noise ratio, so that the infrared image collected based on the adjusted integration time has a higher signal-to-noise ratio. The embodiment of the present invention adjusts the integration time of the photoelectric detection system based on the determined image complexity and signal-to-noise ratio, and can improve the target detection and tracking distance, distance limit, and target tracking stability without changing the target tracking algorithm.

Description

Technical Field

[0001] The present invention relates to the technical field of optoelectronic systems, and in particular to an infrared image acquisition method and device with adjustable integration time. Background Art

[0002] In the process of long-range target tracking or detection performed by the long-range optoelectronic detection system on a high-speed mobile platform, as the target distance, infrared detector temperature and power-on time change, especially when the long-range target being tracked is in a state of flying away from the detection system, that is, the distance between the trailing target and the optical receiving window is getting farther and farther, the signal-to-noise ratio is getting lower and lower. The existing target tracking and prediction algorithms usually increase the minimum signal-to-noise ratio threshold of the algorithm, but no matter how high-performance the signal processing algorithm is, it will lose the tracking target after a certain signal processing threshold. Summary of the invention

[0003] The embodiments of the present invention provide an infrared image acquisition method and device with adjustable integration time, which can improve the target detection and tracking distance and the tracking stability of the target with a range limit without changing the target tracking algorithm.

[0004] The embodiment of the present invention provides an infrared image acquisition method with adjustable integration time, comprising:

[0005] Configure an initial integration time based on the working environment, and collect infrared images at the initial integration time;

[0006] Using the photoelectric detection system, the target is tracked based on the acquired infrared images;

[0007] During the target tracking process:

[0008] Determining the image complexity of a local image where a target exists according to the acquired infrared image; and

[0009] Determine the signal-to-noise ratio of the local image where the target exists;

[0010] The integration time of the infrared detector of the photoelectric detection system is adjusted based on the image complexity and the signal-to-noise ratio, so that the infrared image collected based on the adjusted integration time has a higher signal-to-noise ratio.

[0011] In some embodiments, the infrared image is acquired by receiving radiation from the target via an infrared detector of a photoelectric detection system;

[0012] The image complexity of the local image where the target exists is determined based on the acquired infrared image, including:

[0013] Using a target neighborhood range of a preset size in the infrared image data as the local image;

[0014] Calculate the definition parameter of the sub-image area to satisfy

[0015]

[0016] Among them, E r is the mean value of the target area, E B is the mean value of the background area, δ B is the standard deviation of the background area, and the size of the background area is selected to be 5-10 times the size of the target area.

[0017] In some embodiments, the target neighborhood range of a preset size is determined by an image frame of a preset size centered on the target, or is determined by adjustment based on the scale of the target.

[0018] In some embodiments, adjusting the integration time of the photodetection system based on the image complexity and the signal-to-noise ratio comprises:

[0019]

[0020] Wherein, P1 represents the first boundary value, P2 represents the second boundary value, LCD represents the image complexity of the local image where the target exists, INT(t) represents the evaluation parameter for adjusting the integration time, and Δt represents the step value of the integration time.

[0021] In some embodiments, adjusting the integration time of the photodetection system based on the image complexity and the signal-to-noise ratio further comprises:

[0022] Predict the change in distance of the target relative to the optoelectronic device;

[0023] The integration time of the photodetection system is adjusted based on the distance variation, the image complexity and the signal-to-noise ratio.

[0024] In some embodiments, it also includes:

[0025] A user interface is provided, and the integration time adjustment process is presented to the user through the user interface based on the collected infrared images.

[0026] In some embodiments, the infrared image acquisition method further includes:

[0027] In the case of tracking multiple targets using an optoelectronic detection system, determining image complexity of partial images of multiple existing targets;

[0028] Weighting the image complexity of each local image;

[0029] The integration time of the photodetection system is adjusted based on the weighted result.

[0030] In some embodiments, the infrared image acquisition method further includes:

[0031] Scan and record the location information of each target;

[0032] When the target appears, the photoelectric detection system is controlled based on the position information of the target and the integration time determined in advance.

[0033] An embodiment of the present invention also proposes an image acquisition device with adjustable integration time, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the infrared image acquisition method with adjustable integration time described in the aforementioned embodiments.

[0034] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the infrared image acquisition method with adjustable integration time described in the aforementioned embodiments are implemented.

[0035] The embodiment of the present invention determines the image complexity of the local image where the target exists according to the collected infrared image, determines the signal-to-noise ratio of the local image where the target exists, and adjusts the integration time of the photoelectric detection system based on the image complexity and the signal-to-noise ratio during the process of tracking the target. Thus, the target detection and tracking distance, the distance limit and the tracking stability of the target can be improved without changing the target tracking algorithm.

[0036] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0038] Figure 1 A schematic diagram of the basic flow of the infrared image acquisition method according to an embodiment of the present invention;

[0039] Figure 2 A schematic diagram of the basic structure of a photoelectric detection system according to an embodiment of the present invention;

[0040] Figure 3The figure is a schematic diagram of the overall process of the infrared image acquisition method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0042] In the process of long-range photoelectric detection systems such as airborne photoelectric radars or airborne search and tracking systems performing long-range target tracking or detection, when the long-range target is farther and farther away from the photoelectric detection equipment, the signal-to-noise ratio of the target is getting lower and lower, and an effective automatic focusing technology is needed to replace manual focusing or global focusing. Generally, aerial targets are all point target detection at the level of demonstrating the farthest distance. For this reason, there is the following formula:

[0043]

[0044] Among them, δ is the signal process factor, τ a and τ0 are the atmospheric transmittance and the optical system transmittance respectively, D0 2 is the optical effective aperture, D * is the detection rate of the detection system, SNR TH is the signal processing signal-to-noise ratio threshold of the detection algorithm, which is determined by the performance of the algorithm. a and b are the pixel sizes of the detector. Δf is the signal bandwidth of the detector. Obviously, once the detection system and the atmospheric environment at that time are determined, the above parameters cannot be changed. I is the difference between the target and background radiation intensity. If you want to increase the maximum detection distance, you can only increase the difference between the target and background radiation intensity by focusing or adjusting the integration time.

[0045] Based on this, the present invention proposes a method for obtaining infrared images at different distances, different target sizes, and different background complexities, such as Figure 1 As shown, including:

[0046] In step S101, an initial integration time is configured based on the working environment, and an infrared image is acquired under the initial integration time. Figure 2As shown, the infrared image acquisition method of the present application first sets the initial integration time of the infrared detector of the photoelectric detection system based on the working environment, and collects the infrared image under this integration time. The integration time and frequency of the infrared detector proposed in this example can be adjusted. The initial integration time setting method of the infrared detector of the photoelectric detection system satisfies that the pixel output average value of the infrared detector is near the half-well during background correction, and based on this, the output values ​​of multiple frames of the detector are averaged, which can further eliminate the readout circuit noise and the detector dark current.

[0047] In step S102, the target is tracked based on the collected infrared image using the photoelectric detection system. Specifically, the tracking mode of the photoelectric detection system can be used to track the target based on the collected infrared image. The target can be tracked using a preset target tracking detection algorithm and the real-time miss amount can be sent to the servo control system so that the target is fixed in the preset image area of ​​the infrared image. In the present disclosure, the preset target tracking detection algorithm can include linear regression, Kalman filtering, prediction based on deep learning, etc., and the specific target tracking algorithm is not limited one by one here.

[0048] If the integration time, usage time, and operating temperature change, or even when the integration time increases, blind flash pixels and fixed image noise will increase. In the process of tracking the target in this embodiment:

[0049] In step S103, the image complexity of the local image where the target exists is determined according to the acquired infrared image; and

[0050] Determine the signal-to-noise ratio of the local image where the target exists.

[0051] Specific image processing requires real-time noise suppression before subsequent data processing and integration time evaluation calculation can be carried out. At this time, the image acquisition and preprocessing unit needs to dynamically perform non-uniformity correction and blind flash correction. In the specific implementation process, an adaptive non-uniformity correction algorithm based on the complexity of the continuous frame scene and a real-time blind flash based on spatiotemporal statistical characteristics can be used to eliminate blind flashes.

[0052] Then, the local complexity LCD of the target and its background of the current frame infrared image (from the image acquisition unit) is calculated in real time, and the target local signal-to-noise ratio LSNR of the current frame infrared image is calculated simultaneously.

[0053] Specific

[0054] In step S104, the integration time of the photoelectric detection system is adjusted based on the image complexity and the signal-to-noise ratio, so that the infrared image collected based on the adjusted integration time has a higher signal-to-noise ratio.

[0055] The embodiment of the present invention determines the image complexity of the local image where the target exists according to the collected infrared image, determines the signal-to-noise ratio of the local image where the target exists, and adjusts the integration time of the photoelectric detection system based on the image complexity and the signal-to-noise ratio during the process of tracking the target. Thus, the target detection and tracking distance, the distance limit and the tracking stability of the target can be improved without changing the target tracking algorithm.

[0056] In some embodiments, the infrared image is obtained by receiving the infrared optical system radiation through an infrared detector. The infrared detector referred to in this example may include digital and analog output infrared detectors, cooling and non-cooling detectors, infrared detectors such as mercury cadmium telluride, indium antimonide, vanadium oxide, and graphene, and may also be CMOS, CCD, spectrometer, radar imaging, spectrometer, etc. Determining the image complexity of the local image where the target exists according to the collected infrared image includes: taking the target neighborhood range of a preset size in the infrared image data as the local image. In some embodiments, the target neighborhood range of a preset size is determined by an image frame of a preset size centered on the target, or is adjusted and determined based on the scale of the target. For example, when the target is small, the neighborhood range of the target can be 20*20 pixels as the local image. For another example, when the target is large, 80*80 pixels can be used as the local image. Independently or additionally, a size mapping relationship with the local image range can be set based on the size of the target, so that a suitable local image can be determined as the scale of the target in the infrared image changes. In some examples, the calculation of the local complexity LCD of the infrared image can be implemented by using the local variance method, which has low computational complexity and good real-time performance. In particular, when calculating the local variance in this embodiment, the infrared image used is the 16-bit original energy distribution data output by the infrared detector, rather than the energy stretched data such as the histogram.

[0057] Then, the definition parameter of the sub-image area is calculated to satisfy

[0058]

[0059] Among them, E r is the mean value of the target area, E B is the mean value of the background area, δ Bis the standard deviation of the background area, and the size of the background area is selected to be 5-10 times the size of the target area. As a specific example, the size of the background area can be selected to be 8 times the size of the target area. Specifically, the control amount of the integration time can be determined according to the scene complexity, signal-to-noise ratio, and local average brightness. These parameters specifically refer to the parameter quantities of the target neighborhood to be tracked. No infrared image detection is performed, and special distinction is required. The target neighborhood can use a 64*64 neighborhood range, or it can be adaptively changed according to the target scale, generally 8 times the target size. The position of the target neighborhood is the center of the image before and after tracking. After detection, the detected target centroid is used as the center and its neighborhood is considered.

[0060] In some embodiments, adjusting the integration time of the photodetection system based on the image complexity and the signal-to-noise ratio comprises:

[0061]

[0062] Wherein, P1 represents the first boundary value, P2 represents the second boundary value, LCD represents the image complexity of the local image where the target exists, INT(t) represents the evaluation parameter for adjusting the integration time, and Δt represents the step value of the integration time. The evaluation parameter for adjusting the integration time thus calculated can be used to adjust the integration time of the photoelectric detection system. In some examples, if the evaluation parameter is out of range, the integration time adjustment mechanism can be changed to start. For example, if the increase in the integration time still cannot effectively acquire the target and achieve stable tracking, the integration time can be further increased by reducing the acquisition frame rate.

[0063] In some embodiments, adjusting the integration time of the photodetection system based on the image complexity and the signal-to-noise ratio further comprises:

[0064] Predict the change in the distance of the target relative to the photoelectric device. Specifically, Kalman filtering can be used to predict the change in the distance of the target relative to the photoelectric device.

[0065] The integration time of the photoelectric detection system is adjusted based on the distance variation, the image complexity and the signal-to-noise ratio. The corresponding integration time can be sent to the photoelectric detector in combination with the target evaluation parameter.

[0066] In some embodiments, it also includes:

[0067] A user interface is provided, and the integration time adjustment process is presented to the user through the user interface based on the collected infrared image. The real-time target local complexity, local signal-to-noise ratio and target distance are displayed in a visual interface to assist the operator in judging the integration time adjustment quality of the photoelectric detection system. The visual user interface can also provide the integration time performance index, target distance and infrared real-time image, wherein the infrared real-time image can be converted into an enhanced image or a pseudo-color image according to actual needs.

[0068] In some embodiments, the infrared image acquisition method further includes:

[0069] In the case of tracking multiple targets using a photoelectric detection system, the image complexity of multiple local images where the targets exist is determined; the image complexity of each local image is weighted; and the integration time of the photoelectric detection system is adjusted based on the weighted result.

[0070] Specifically, in this example, the photoelectric detection system uses a laser ranging module to measure the distance of key targets and calculates the target imaging scale through infrared images. When multiple targets appear in the field of view or when executing a task, the local complexity and local signal-to-noise ratio of multiple targets and the surrounding background can be weighted to adjust the integration time.

[0071] In some embodiments, the infrared image acquisition method further includes: scanning and recording the position information of each target; when the target appears, controlling the photoelectric detection system based on the position information of the target and a previously determined integration time.

[0072] Specifically, the servo system can be controlled to scan multiple target positions and record them. When a target appears, the integration time value of the previous position can be read to control the integration time of the photoelectric detector. Optionally, if the integration time is adjusted to the maximum and the target energy cannot be effectively obtained, the target detection and tracking distance can be improved by reducing the frame rate of the detector and then further increasing the integration time.

[0073] The present invention also proposes a specific implementation of an infrared image acquisition method with adjustable integration time. Figure 3 As shown:

[0074] In step S301, an initial integration time is set based on the working environment, and infrared images are collected under this integration time. The real-time target local complexity, local signal-to-noise ratio and target distance are displayed on a visual interface to assist the operator in determining whether the integration time is appropriate for the scene where the target is located.

[0075] In step S302, target detection is performed. When the mode is switched to target tracking, the target is tracked by a target tracking detection algorithm so that the target is fixed in a preset area.

[0076] In step S303, the local complexity of each current frame image is calculated in real time. In order to improve the real-time performance of the system, the local complexity is calculated using the variance between the target and its surrounding area. The neighborhood range of the target is 64*64 pixels.

[0077] In step S304, the evaluation parameters are comprehensively determined based on complexity, signal-to-noise ratio, and average brightness illumination, and the target is fine-tuned in the forward and reverse directions. When the algorithm detects and tracks the target and a certain signal-to-noise ratio is reached, the integration time change is stopped.

[0078] In step S305, a tracking algorithm such as Kalman filtering or kernel correlation filtering is used to predict the change in the distance of the target relative to the optoelectronic device, so as to estimate the increase or decrease of the integration time in advance.

[0079] In step S306, the real-time key parameters are superimposed on the infrared real-time image and displayed on the visualization interface.

[0080] When the target distance or scene complexity changes, the operations S303-S306 are executed in a loop until target detection or other image processing algorithms are performed when the target is lost.

[0081] When there are multiple targets of interest, the local complexity of the multiple targets is weighted to determine the value of this integration time.

[0082] The infrared image acquisition method proposed in this embodiment adopts the method of adjusting the integration time in real time by tracking the target, which can effectively improve the clarity of the target image. When the target distance is getting farther and farther, the method of the present invention can improve the radiation intensity difference between the target and the background without changing the target detection and tracking algorithm, thereby improving the signal-to-noise ratio of the target, and finally improving the farthest detection and tracking distance, and improving the tracking stability of the extreme target. The method of adjusting the integration time in real time by tracking the target and the image acquisition device reduce the user's subjective judgment during the flight, reduce the process of manually changing the integration time, make the operation simple, and also improve flight safety. The method of this embodiment can be combined with a laser ranging system to verify the integration time of targets at different distances, and can also be used to generate a distance integration time database. After actual use, the integration time of aerial targets at different distances can be set in advance from 1km to 100km.

[0083] An embodiment of the present invention also proposes an image acquisition device with adjustable integration time, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the infrared image acquisition method with adjustable integration time described in the aforementioned embodiments.

[0084] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the infrared image acquisition method with adjustable integration time described in the aforementioned embodiments are implemented.

[0085] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0086] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0087] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for a terminal (which can be a mobile phone, computer, server or network equipment, etc.) to execute the methods described in each embodiment of the present invention.

[0088] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A method for acquiring an infrared image with adjustable integration time, characterized in that: include: Configure an initial integration time based on the working environment, and collect infrared images at the initial integration time; Using the photoelectric detection system, the target is tracked based on the acquired infrared images; During the target tracking process: Determine the image complexity of a local image where a target exists according to the acquired infrared image; as well as Determine the signal-to-noise ratio of the local image where the target exists; The integration time of the infrared detector of the photoelectric detection system is adjusted based on the image complexity and the signal-to-noise ratio, so that the infrared image collected based on the adjusted integration time has a higher signal-to-noise ratio.

2. The infrared image acquisition method with adjustable integration time as claimed in claim 1, characterized in that: The infrared image is obtained by receiving radiation from the target through the infrared detector of the photoelectric detection system; The image complexity of the local image where the target exists is determined based on the acquired infrared image, including: Using a target neighborhood range of a preset size in the infrared image data as the local image; Calculate the definition parameter of the sub-image area to satisfy Among them, E r is the mean value of the target area, E B is the mean value of the background area, δ B is the standard deviation of the background area, and the size of the background area is selected to be 5-10 times the size of the target area.

3. The infrared image acquisition method with adjustable integration time as claimed in claim 2, characterized in that: The target neighborhood range of the preset size is determined by an image frame of the preset size centered on the target, or is determined by adjustment based on the scale of the target.

4. The infrared image acquisition method with adjustable integration time as claimed in claim 2, characterized in that: Adjusting the integration time of the photoelectric detection system based on the image complexity and the signal-to-noise ratio includes: Among them, P1 represents the first boundary value, P2 represents the second boundary value, LCD represents the image complexity of the local image with the target, INT(t) represents the evaluation parameter of the adjusted integration time, and Δt represents the step value of the integration time.

5. The infrared image acquisition method with adjustable integration time as claimed in claim 4, characterized in that: Adjusting the integration time of the photoelectric detection system based on the image complexity and the signal-to-noise ratio also includes: Predict the change in distance of the target relative to the optoelectronic device; The integration time of the photodetection system is adjusted based on the distance variation, the image complexity and the signal-to-noise ratio.

6. The infrared image acquisition method with adjustable integration time as claimed in claim 2, characterized in that: Also includes: A user interface is provided, and the integration time adjustment process is presented to the user through the user interface based on the collected infrared images.

7. The infrared image acquisition method with adjustable integration time as claimed in claim 1, characterized in that: The infrared image acquisition method further comprises: In the case of tracking multiple targets using an optoelectronic detection system, determining image complexity of partial images of multiple existing targets; Weighting the image complexity of each local image; The integration time of the photodetection system is adjusted based on the weighted result.

8. The infrared image acquisition method with adjustable integration time as claimed in claim 7, characterized in that: The infrared image acquisition method further comprises: Scan and record the location information of each target; When the target appears, the photoelectric detection system is controlled based on the position information of the target and the integration time determined in advance.

9. An image acquisition device with adjustable integration time, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the infrared image acquisition method with adjustable integration time are implemented as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the infrared image acquisition method with adjustable integration time as described in any one of claims 1 to 8 are implemented.

Images