A method and system for imaging distortion correction for thermal imaging devices

Abstract

This invention relates to an imaging distortion correction method and system for thermal imaging equipment. The method includes: addressing the image geometric distortion problem caused by irregularly shaped curved infrared windows in dynamic environments, firstly, establishing a quantitative mapping model from wavefront aberration to image geometric distortion in a ground-based laboratory; during equipment operation, sensing and calculating the wavefront distortion transmitted through the window in real time to predict the aberration state at future moments; then, inputting the predicted aberration into the mapping model to calculate the corresponding image distortion field, and combining it with the equipment's static distortion to generate a dynamic pixel remapping lookup table; finally, using this lookup table to remap and interpolate the real-time acquired thermal image stream, outputting a geometrically corrected image with pre-compensated dynamic aberrations; this method achieves forward-looking correction of dynamic aberrations, significantly improving the geometric fidelity, stability, and environmental adaptability of thermal imaging under complex conditions, providing high-quality input for subsequent target recognition and tracking.

Description

Technical Field

[0001] This invention relates to the field of computer vision image processing technology, and in particular to an imaging distortion correction method and system for thermal imaging equipment. Background Technology

[0002] Thermal imaging equipment equipped with irregularly shaped curved infrared windows faces severe challenges in dynamic application platforms such as high-speed aircraft. During high-speed maneuvers, the intense aerodynamic heating effects and structural loads cause complex deformations in the infrared window, introducing dynamic optical aberrations that change rapidly over time. This aberration leads to distortion of the wavefront of the transmitted infrared beam, ultimately manifesting as severe, non-uniform image geometric distortion on the detector's focal plane. Traditional static calibration and correction methods can only compensate for the inherent, fixed lens distortion of the equipment, and are completely unable to cope with this dynamic distortion that is strongly correlated with flight conditions. As a result, in dynamic environments, the geometric accuracy of the images output by the thermal imaging system decreases sharply, and the shape and position of the target become distorted and drifted, severely restricting the performance and reliability of subsequent image-based automatic target recognition, precise tracking, and guidance tasks.

[0003] To address dynamic aberrations, existing technologies attempt to introduce active optical elements such as deformable mirrors into the optical path for real-time wavefront correction. However, such systems are complex, expensive, and struggle to achieve sufficient bandwidth and stability in highly dynamic environments. Another approach is post-processing image correction using algorithms, but this typically relies on simplified assumptions about aberration mechanisms or offline simulation models, failing to adapt to the complex and variable physical conditions of real-world flight, making it difficult to balance real-time performance and accuracy. The core bottleneck lies in the lack of an effective technical approach capable of real-time and accurate sensing and prediction of dynamic wavefront distortion caused by the window during flight, and directly and rapidly converting this optical information into pixel-level geometric correction values ​​for the image. Summary of the Invention

[0004] This invention addresses the technical problems existing in the prior art by tightly coupling optical correction and image processing to achieve a proactive, high-precision real-time correction method and system for dynamic geometric distortion.

[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: S1. For thermal imaging devices equipped with irregularly shaped curved infrared windows, a quantitative mapping model from wavefront aberration coefficients to image geometric distortion fields is trained in a ground-based laboratory by actively injecting known infrared wavefront aberrations and simultaneously acquiring target thermal images. S2. The motion trajectory of the device during operation is recorded, and the wavefront distortion after passing through the infrared window is sensed in real time using a beam splitter, and calculated into Zernike aberration coefficients that can be used to describe the wavefront distortion, serving as the real-time causal input for the quantitative mapping model. S3. The real-time wavefront aberration coefficients of the device are fused with historical aberration data to predict the wavefront aberration coefficients that will appear in the thermal imaging optical path at a fixed time in the future. S4. The predicted future aberration coefficients are input into the quantitative mapping model to calculate the corresponding future thermal image distortion field. Combined with the static distortion of the device, a dynamic pixel remapping lookup table for real-time image geometric correction is generated. S5. The lookup table is used to perform pixel relocation and interpolation operations on the real-time acquired raw thermal image stream, outputting a frame of thermal image corrected in advance for future time.

[0006] Preferably, the calculation of the Zernike aberration coefficients that can be used to describe the wavefront aberration includes: splitting the incident infrared beam passing through the irregular curved window to obtain a beam for wavefront sensing; measuring the sensing beam using a wavefront sensor to obtain data reflecting wavefront distortion; and calculating and outputting the coefficients that describe the current wavefront aberration based on the two measured data in real time.

[0007] Preferably, the step of generating a dynamic pixel remapping lookup table for real-time image geometric correction includes: calling a pre-calibrated static geometric distortion correction map; inputting the predicted future wavefront aberration information into a quantitative mapping model to obtain the corresponding dynamic geometric distortion data; fusing the static geometric distortion correction map with the dynamic geometric distortion data to generate a comprehensive correction map; determining the sub-pixel coordinates and interpolation weights of each pixel position in the corrected image in the original distorted image based on the comprehensive correction map; and generating a lookup table for image processing hardware to perform pixel remapping based on the corresponding coordinates and interpolation weights.

[0008] Preferably, the output of a frame of thermal image with pre-compensated future time correction includes: S51, after obtaining the corrected thermal image, when a potential low-emissivity target is identified in it, triggering a synchronous laser excitation process to acquire an excitation frame and a natural frame; S52, based on the continuity of the low-emissivity target's motion, mapping the clear features in the excitation frame to the natural frame for sub-pixel-level edge localization; S53, in the natural frame where edge localization has been completed, performing signal mixing modeling and calculating the coverage ratio for edge pixels; S54, fusing multiple sets of high-precision edge data and pixel mixing ratios to construct and update the enhanced geometric and radiation characteristic model of the low-emissivity target, and correcting the thermal image of the low-emissivity target in real time.

[0009] Preferably, the triggering synchronous laser excitation process for acquiring excitation frames and natural frames includes: when the thermal imaging device is working, analyzing the obtained corrected image to identify potential low-emissivity target areas; when the potential low-emissivity target area is identified, controlling the integrated pulsed laser to emit laser pulses towards the area, causing a brief thermal mark to form on the target surface; synchronously controlling the infrared detector to acquire images of the target after it has been heated during the laser pulse as excitation frames, and acquiring images of the target in its natural thermal state before and after the laser pulse as natural frames.

[0010] Preferably, the triggering synchronous laser excitation process for acquiring excitation frames and natural frames further includes: S511, determining in real time whether naturally occurring infrared transient events have perceptual value for enhancing low emissivity targets based on the corrected thermal image; S512, when the transient event has perceptual value, acquiring a frame sequence covering the entire event process and extracting the intrinsic thermal features of the target caused by the transient event; S513, based on the extracted intrinsic thermal features of the target, quickly performing reverse calibration of the target's high-precision geometry and radiation characteristics to establish a golden period standard model for the current moment; S514, merging the golden period standard model into the quantitative mapping model to predict the precise contour and theoretical radiation pattern that the low emissivity target should present in the distortion-corrected image of the next frame.

[0011] Preferably, the golden period standard model includes: a first process for instantaneous fine calibration of the target geometry and spatial attitude, including extracting a clear outline of the target from its intrinsic thermal characteristics, registering it with historical predicted outlines, and analytically calculating the target's three-dimensional position, orientation, and outline dimensions in conjunction with the direction of the transient event source; and a second process for reverse calibration of the target's radiation characteristics, including measuring the radiation distribution on the target surface based on the target's intrinsic thermal characteristics, estimating its theoretical irradiance in conjunction with the physical model of the transient event source, and deducing the effective infrared emissivity distribution and basic radiation value of the target surface in reverse through physical relationships.

[0012] Preferably, the prediction of the precise contour of the low-emissivity target in the distortion-corrected image of the next frame includes: defining the target region of interest in the current corrected image, extracting and initializing a persistent set of micro-features, and constructing a joint state vector containing the target motion, aberrations, and feature three-dimensional coordinates; in each processing cycle, further forward prediction of the joint state vector based on the physical model to generate predictions for all observables in the next frame image; after obtaining the new frame corrected image, performing probabilistic correlation of weak signals near the predicted position, and using this soft observation to perform Bayesian update of the joint state vector; when a natural transient event is detected, using high-precision observations to strongly correct and reset the joint state during gliding.

[0013] Preferably, the generation of predictions for all observable quantities in the next frame image includes: predicting the motion state of the target based on the target kinematic model and aerodynamic constraints; predicting the dynamic aberration state based on the window aberration evolution model; and calculating the predicted position of each persistent micro-feature in the original distorted image of the next frame by combining the predicted target motion state and aberration state.

[0014] This application also provides an imaging distortion correction system for a thermal imaging device, the system comprising: The data-driven module is used to train a quantitative mapping model from wavefront aberration coefficients to image geometric distortion field in a ground laboratory for thermal imaging equipment equipped with irregularly curved infrared windows by actively injecting known infrared wavefront aberrations and simultaneously acquiring target thermal images. The measurement and analysis module is used to record the motion trajectory of the device during operation, and uses the beam splitter to sense the wavefront distortion after passing through the infrared window in real time, and calculates it into Zernike aberration coefficients that can be used to describe it, as a real-time input of the cause of thermal image distortion. The analysis and prediction module is used to fuse the real-time wavefront aberration coefficients of the device with historical aberration data to predict the wavefront aberration coefficients that will appear in the thermal imaging optical path at a fixed time in the future. The real-time calculation module is used to input the predicted future aberration system into the quantitative mapping model, calculate the corresponding future thermal image distortion field, and generate a dynamic pixel remapping lookup table for real-time image geometric correction by combining the static distortion of the device. The image processing module is used to perform pixel relocation and interpolation operations on the raw thermal image stream acquired in real time using a lookup table, and output a frame of thermal image that has been pre-compensated for future time correction.

[0015] The beneficial effects of this invention are: This method effectively solves the problem of geometric distortion in thermal images caused by irregularly shaped curved infrared windows in dynamic environments. It achieves proactive compensation for dynamic aberrations through real-time wavefront aberration sensing and prediction, and integrates this with static distortion correction. This method significantly improves the geometric fidelity and stability of thermal imaging output under complex conditions, providing high-quality image input for core target recognition, tracking, and guidance algorithms, and enhancing the overall environmental adaptability and mission reliability of the method.

[0016] By employing synchronized laser excitation and dual-frame comparative analysis, a breakthrough in detection capabilities has been achieved specifically for low-emissivity, small targets. This approach not only significantly improves the detection probability and recognition accuracy of such targets but also effectively decouples them from their true radiation characteristics. Consequently, in complex dynamic environments, it provides more reliable and abundant enhanced information for subsequent target identification, tracking, and precise measurement, comprehensively improving the overall sensing performance of this method.

[0017] Building upon existing active and geometric correction capabilities, this system creatively transforms uncontrollable natural infrared transient events in the environment (such as flashes and crepuscular rays) into valuable opportunities to enhance sensing performance. In a purely passive operating mode, it intelligently identifies and utilizes these brief event windows to achieve a one-time, high-precision calibration of the geometric and radiometric characteristics of low-emissivity targets. This not only reduces reliance on active illumination methods, enhancing the system's stealth and environmental adaptability, but also significantly improves the accuracy and stability of continuous tracking of weak targets during event gaps by establishing a golden period standard model. This comprehensively enhances the resilience of the thermal imaging system in fully passive detection and identification within complex, adversarial environments.

[0018] By jointly estimating target motion, window aberration, and persistent micro-features, autonomous, continuous, and robust tracking of low-observable targets is achieved in the absence of clear external events, significantly extending the maintenance time of high-precision sensing. Simultaneously, it can intelligently utilize any sudden natural events to reset accumulated errors during gliding, thereby providing stable, reliable target state information with explicit confidence assessments in any environment. This greatly enhances the ability to continuously monitor and strike concealed targets in complex adversarial environments. Attached Figure Description

[0019] Figure 1 This is a schematic flowchart of an imaging distortion correction method for a thermal imaging device according to an embodiment of the present invention. Figure 2 This is a structural block diagram of an imaging distortion correction system for a thermal imaging device according to an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0022] In the description of this application, the term "for example" is used to indicate that it is used as an example, illustration, or illustration. Any embodiment described as "for example" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0023] Example 1: Figure 1 This is a flowchart illustrating an imaging distortion correction method for a thermal imaging device according to Embodiment 1 of the present invention, comprising the following steps: S1, for thermal imaging equipment equipped with irregular curved infrared windows, a quantitative mapping model from wavefront aberration coefficients to image geometric distortion field is trained in a ground laboratory by actively injecting known infrared wavefront aberrations and simultaneously acquiring target thermal images.

[0024] The active injection of known infrared wavefront aberrations is achieved by inserting a deformable mirror into the main optical path. The deformable mirror is controlled by a computer to generate known wavefront aberration patterns described by Zernike polynomial coefficients [Z1,Z2,...,Z15].

[0025] Specifically, on an aerodynamic-thermal-structural coupling test platform, an irregularly shaped curved infrared window is installed, and a series of known wavefront aberrations are actively injected through a deformable mirror. Wavefront sensor data after aberration injection (recording the actual wavefront phase distribution of the beam) and target thermal images on the main infrared detector (recording the geometric deformation of a known standard pattern due to aberrations) are simultaneously acquired. The target thermal images are analyzed to calculate the geometric distortion displacement field compared to the ideal position. A quantitative mapping model is trained using the acquired wavefront aberration coefficients and corresponding geometric distortion field data pairs.

[0026] The geometric distortion displacement field is a two-dimensional matrix with the same size as the resolution of the main infrared image. Each element in the matrix stores a two-dimensional displacement vector (Δx, Δy), representing the coordinate offset of the pixel position due to aberration.

[0027] S2 records the motion trajectory of the device during operation, uses the beam splitter to sense the wavefront distortion behind the infrared window in real time, and solves it into Zernike aberration coefficients that can be used to describe it, serving as the real-time causal input for the quantitative mapping model.

[0028] Specifically, when the thermal imaging device is working, the incident infrared beam passing through the irregularly shaped curved window is split, with one path used for imaging and the other for wavefront sensing. The wavefront sensor measures the wavefront slope within the sub-aperture through a microlens array, which is represented as the displacement of the light spot array. Based on the displacement of the light spot array, the Zernike aberration coefficients [Z(t)] of the current wavefront are calculated in real time using a wavefront reconstruction algorithm. The calculated wavefront aberration coefficients [Z(t)] are continuously output at a high frequency (defined as greater than or equal to 1000 Hz).

[0029] Among them, the wavefront slope is the degree of tilt of the wavefront phase in a local region. Each microlens corresponds to a sub-aperture, and the average wavefront slope within the sub-aperture causes its focused spot to shift laterally on the detector.

[0030] S3, by fusing the real-time wavefront aberration coefficients of the device with historical aberration data, predicts the wavefront aberration coefficients that will appear in the thermal imaging optical path at a fixed time in the future.

[0031] Specifically, the system continuously receives the current aberration coefficient sequence calculated in real time from the wavefront sensor, while simultaneously acquiring device operating status parameters synchronized with the aberration data, such as the carrier's motion parameters and the physical state parameters of the window region. This real-time data, along with multiple consecutive frames of aberration and status data within the most recent historical time window, constitutes a time series sample. This time series is fed into a lightweight time-series prediction model trained on extensive experimental data. This model internalizes the correlation between aberration dynamics and operating status, and by calculating the input sequence, outputs a set of Zernike coefficients. These coefficients represent the best estimate of the wavefront aberrations that the imaging optical path will exhibit after a fixed processing delay in the future. This prediction aims to compensate for the inherent time delay in the entire signal chain from aberration perception to image correction, enabling the correction action to be based on future predictions, thus achieving proactive compensation.

[0032] For example, suppose the current time is t, and the device caches 10 sets of historical aberration coefficients [Z] and corresponding flight states at times t, t-1ms, t-2ms, ..., t-9ms. The prediction model receives these 10 sets of time-series data, performs calculations through its internal network, and outputs a set of Zernike coefficients, which is the predicted value of the wavefront aberration at time t+5ms [Z_pred(t+5ms)]. The prediction model is a lightweight recurrent neural network (RNN) or one-dimensional convolutional network (1D-CNN) pre-trained on the ground using extensive simulation and experimental data. It is used to learn the implicit physical relationship between dynamic changes in aberrations and the carrier's motion and thermodynamic state.

[0033] S4. Input the predicted future aberration system into the quantitative mapping model to calculate the corresponding future thermal image distortion field. Combined with the static distortion of the device, generate a dynamic pixel remapping lookup table for real-time image geometric correction.

[0034] The generation of the dynamic pixel remapping lookup table begins by calling the device's inherent static distortion correction data. This data is a pre-calibrated and stored reference mapping used to correct geometric deviations of the lens assembly itself. Once the processing unit obtains the predicted wavefront aberration for future moments, it immediately calls the ground-trained mapping model to convert the predicted value into a predicted offset map describing the dynamic geometric distortion of the entire image. The static correction mapping and the dynamic predicted offset map are then superimposed at the pixel level to obtain a total remapping field that integrates both static and dynamic distortions. Based on this total remapping field, for each integer pixel coordinate of the corrected image, its corresponding sub-pixel coordinate in the original distorted image is precisely calculated, and weighting coefficients for grayscale interpolation are determined. Finally, these correspondences are compiled and optimized into a lookup table that can be directly read and executed by the image processing hardware.

[0035] S5 uses a lookup table to perform pixel relocation and interpolation operations on the raw thermal image stream acquired in real time, and outputs a frame of thermal image that has been pre-compensated for future time correction.

[0036] The pixel relocation and interpolation operations utilize the newly generated dynamic pixel remapping lookup table. For each preset integer coordinate position in the corrected output image, the corresponding source coordinate data is retrieved according to the coordinate index lookup table. This data indicates which sub-pixel coordinate position in the current input original thermal image should be sampled from and includes weighting coefficients for interpolation calculation. Based on the integer part of the source coordinates, the gray values ​​of the four nearest pixels are quickly read from the memory of the original image. Using the weighting coefficients provided by the lookup table, bilinear interpolation is performed on the gray values ​​of these four pixels to calculate the optimal, geometrically corrected gray value corresponding to the output pixel position. This process processes all pixels of the entire frame in parallel in a pipelined manner. All the calculated pixel gray values ​​are combined sequentially to output a complete thermal image frame whose geometry has been pre-compensated for dynamic aberrations at future moments.

[0037] The technical solutions described in the embodiments of this application above have at least the following technical effects or advantages: This method effectively solves the problem of geometric distortion in thermal images caused by irregularly shaped curved infrared windows in dynamic environments. It achieves proactive compensation for dynamic aberrations through real-time wavefront aberration sensing and prediction, and integrates this with static distortion correction. This method significantly improves the geometric fidelity and stability of thermal imaging output under complex conditions, providing high-quality image input for core target recognition, tracking, and guidance algorithms, and enhancing the overall environmental adaptability and mission reliability of the method.

[0038] Example 2: In Example 1, the global dynamic geometric distortion caused by irregular curved windows was effectively corrected through a wavefront sensing, prediction, and mapping model, solving the core problem of image distortion caused by optical windows. However, this scheme mainly ensures the optical accuracy of the imaging system, and does not address the fundamental problem of low imaging contrast for the target's own characteristics—especially for elusive targets with extremely low emissivity and weak, almost transparent signals in thermal images. When the difference between such targets and the background thermal radiation is slight, the image signal-to-noise ratio is extremely low, and even with perfect geometric correction, significant difficulties remain in subsequent target detection, recognition, and accurate measurement. To address this challenge, it is necessary to introduce enhancement mechanisms for specific targets while maintaining the global geometric correction effect.

[0039] Therefore, based on the calibration pipeline established in Example 1, Example 2 integrates active laser excitation and synchronous analysis technology to specifically enhance the detection and analysis capabilities for weak targets with low emissivity.

[0040] In some embodiments, in step S5, outputting a frame of thermal image that has been pre-compensated for future time correction includes: S51, after obtaining the corrected thermal image, when a potential low-emissivity target is identified in it, the synchronous laser excitation process is triggered to acquire excitation frames and natural frames.

[0041] Among them, the synchronous laser excitation process refers to emitting a laser towards a low emissivity target in a very short time and strictly synchronizing the acquisition of two frames (excitation frame and natural frame) with different thermal states.

[0042] Specifically, a target detection algorithm is run within the continuously received globally calibrated thermal image stream. When a region is detected with a signal-to-noise ratio below a set threshold, a shape matching the target characteristics, but extremely low thermal contrast, it is initially identified as a potential low-emissivity target, and its approximate region is locked. Control commands are sent to a pulsed laser integrated on the same platform as the thermal imaging equipment, with its optical axis calibrated. The laser emits a short laser beam with a wavelength within the sensitive band of the infrared detector and a pulse width in the nanosecond to microsecond range towards the locked target region. The laser energy is absorbed by the target surface, causing a slight, instantaneous temperature rise in the local area within the microsecond range, thus forming a brief but highly contrasting thermal mark on the target against the cold background. The exposure sequence of the main infrared detector is strictly controlled synchronously, acquiring two frames of images in a trigger mode higher than the imaging frame rate: one frame is the excitation frame, exposed during the period when the laser pulse completely covers the target, in which the target has a clear outline due to the thermal mark; the other frame is the natural frame, exposed before or immediately after the laser pulse is emitted, in which the target is in its original low thermal contrast state.

[0043] During this synchronous acquisition process, the wavefront sensor in Example 1 is ensured to work synchronously, and the wavefront aberration data at the precise moments of the acquisition excitation frame and natural frame are recorded.

[0044] S52, based on the continuity of motion of low-emissivity targets, maps sharp features in the excitation frame to the natural frame for sub-pixel-level edge localization.

[0045] The continuity of motion refers to the fact that the target's movement and deformation are minute and continuous within the millisecond-level time interval between the excitation frame and the natural frame acquisition, thus its spatial position is strongly correlated. Sub-pixel-level edge localization refers to obtaining target contour position information with an accuracy better than one pixel.

[0046] S53, in natural frames where edge localization has been completed, performs signal mixing modeling and calculates the coverage ratio for edge pixels.

[0047] Signal fusion modeling refers to modeling the infrared radiation signal received by a single pixel as a linear weighted sum of target radiation and background radiation. Coverage ratio ( () refers to the proportion of the area covered by the target object within the photosensitive area of ​​a pixel, with a value between 0 and 1.

[0048] Specifically, based on the sub-pixel precision predicted contour lines obtained in natural frames, the set of pixels traversed by the target edge is determined. These pixels are called edge pixels. For each edge pixel i, its observed grayscale value... Satisfy the following physical model: ,in, It is the average radiation intensity (grayscale) of the target surface. It is the average radiation intensity (grayscale) of the background surrounding the target. is the coverage ratio of the target within that pixel, and n(i) is the noise. In the excitation frame, the grayscale of low emissivity target areas is enhanced by heating radiation, and its grayscale can be approximated as... ΔI is estimated by the laser energy. This is achieved by comparing the pure pixels within the target in the excitation frame (…). ≈1) and background pure pixels in natural frames ( Given a grayscale value approximately equal to 0, combined with ΔI, a simultaneous equation can be used to estimate the grayscale value. and An approximate value. The estimated... and and the measured grayscale of edge pixels in natural frames. By substituting the values ​​into the hybrid model, the coverage ratio of each edge pixel can be calculated. .

[0049] S54 integrates multiple sets of high-precision edge data and pixel mixing ratios to construct and update an enhanced geometric and radiometric model of low-emissivity targets, thereby correcting the thermal images of low-emissivity targets in real time.

[0050] The enhancement model includes a model describing the target's precise geometry and spatial attitude, as well as a model that calculates the target's true radiometric value from mixed pixel grayscale. Real-time correction refers to feeding back this model information to improve the imaging and measurement quality of the same target in subsequent frames.

[0051] The technical solutions described in the embodiments of this application above have at least the following technical effects or advantages: By employing synchronized laser excitation and dual-frame comparative analysis, a breakthrough in detection capabilities has been achieved specifically for low-emissivity, small targets. This approach not only significantly improves the detection probability and recognition accuracy of such targets but also effectively decouples them from their true radiation characteristics. Consequently, in complex dynamic environments, it provides more reliable and abundant enhanced information for subsequent target identification, tracking, and precise measurement, comprehensively improving the overall sensing performance of this method.

[0052] Example 3: In Example 2, low-emissivity targets were illuminated by actively emitted lasers to artificially create thermal markers, thereby enhancing the perception and high-precision calibration of their geometric and radiation characteristics. However, the effective implementation of this method relies on the active emission energy of the user. In tactical scenarios emphasizing covert penetration or electronic silence, laser emission would expose the platform's position, posing significant tactical risks. Furthermore, the effective range, timing, and energy of laser excitation are strictly constrained by hardware performance, making it difficult to deal with rapidly appearing, multiple, or excessively distant targets, fundamentally limiting its application scope and initiative. To maintain enhanced detection capabilities against low-observable targets while completely eliminating the exposure risks of active emission and expanding the universality of its application scenarios, it is necessary to seek a novel, completely passive enhancement mechanism that does not rely on active illumination.

[0053] In some embodiments, in step S51, triggering the synchronous laser excitation process to acquire excitation frames and natural frames includes: S511 determines in real time, based on the corrected thermal image, whether naturally occurring infrared transient events have the value of enhancing the perception of low-emissivity targets.

[0054] Infrared transient events refer to naturally occurring phenomena in the environment that are short-lived (milliseconds to seconds) and cause rapid and significant changes in local infrared radiation intensity, such as explosive flashes, sudden sunlight through gaps in clouds, or afterburners from the engines of low-emissivity targets. Perception value is determined by whether the spatial location of the event is on the currently tracked low-emissivity target itself, or within the line of sight between the target and the detector, potentially indirectly illuminating the target; and whether the estimated radiation intensity of the event is sufficient to produce a noticeable and detectable temperature rise or reflection signal on the target surface. Only when the spectral characteristics of the event are valid, its spatial location is highly correlated with the target, and the estimated illumination intensity is sufficient, is the event ultimately deemed to have high value for enhancing target perception.

[0055] S512: When a transient event has perceived value, acquire a frame sequence covering the entire process of the event and extract the intrinsic thermal features of the target caused by the transient event.

[0056] Among them, the intrinsic thermal characteristics of the target refer to the thermal radiation distribution and clear outline that are purely excited or revealed on a low emissivity target by the transient event after removing irrelevant factors such as background, carrier motion and window aberration.

[0057] S513, based on the extracted intrinsic thermal features of the target, quickly performs reverse calibration of the target's high-precision geometry and radiation characteristics, and establishes a standard model of the golden period at the current moment.

[0058] The standard model for the golden period includes the target's precise spatial position and attitude at the current moment, its detailed three-dimensional profile, and the distribution of effective infrared radiation characteristics on its surface. Its accuracy is far higher than the estimates in the conventional tracking mode.

[0059] Specifically, the golden age standard model involves two calibration processes. The first process is the instantaneous fine calibration of the target's geometry and spatial attitude. From the acquired intrinsic thermal feature image, a sharp outline of the target is extracted with accuracy superior to that of a single pixel. This sharp outline is then compared and registered with the target outline predicted by conventional tracking before the event. Combining this with the known direction of the transient event light source, a spatial geometric analytical algorithm is used to quickly calculate the target's three-dimensional spatial position, orientation, and outline dimensions that optimally represent the extracted sharp outline. This calculation result is established as the gold standard for the target's geometric state. The second process is the reverse calibration of the target's radiation characteristics. From the intrinsic thermal feature image, the radiation intensity of different regions of the target surface under the event's peak illumination is measured. Simultaneously, based on a physical model of the event source (e.g., considering the sun as a blackbody radiation source of known temperature), the theoretical radiation energy reaching the target surface is estimated. According to the laws of infrared physics, the measured intensity is correlated with the theoretical irradiance, and the effective infrared emissivity distribution map of each region of the target surface is derived in reverse. This distribution map, together with the recalibrated base radiation value of the target itself, constitutes the gold standard for the target's radiation characteristics.

[0060] S514 incorporates the standard model of the golden period into the quantitative mapping model to predict the precise outline and theoretical radiation pattern that low-emissivity targets should present in the next frame of the distortion-corrected image.

[0061] The technical solutions described in the embodiments of this application above have at least the following technical effects or advantages: Building upon existing active and geometric correction capabilities, this system creatively transforms uncontrollable natural infrared transient events in the environment (such as flashes and crepuscular rays) into valuable opportunities to enhance sensing performance. In a purely passive operating mode, it intelligently identifies and utilizes these brief event windows to achieve a one-time, high-precision calibration of the geometric and radiometric characteristics of low-emissivity targets. This not only reduces reliance on active illumination methods, enhancing the system's stealth and environmental adaptability, but also significantly improves the accuracy and stability of continuous tracking of weak targets during event gaps by establishing a golden period standard model. This comprehensively enhances the resilience of the thermal imaging system in fully passive detection and identification within complex, adversarial environments.

[0062] Example 4: Example 3 relies on randomly occurring infrared transient events in the environment for high-precision calibration of low-observable targets, but its effectiveness is limited by the uncontrollability of these events. During long time intervals without effective events, the target tracking status continuously degrades due to the lack of clear observations, posing a risk of loss. To maintain high-precision, continuous, and stable tracking of low-observable targets even in the absence of natural events, it is necessary to develop a capability that does not rely on clear external signals but utilizes the target's weakest and most persistent intrinsic characteristics for autonomous and robust state estimation.

[0063] In some embodiments, in step S514, predicting the precise outline that the low emissivity target should present in the next frame of the distortion-corrected image includes: 4a. Define the target region of interest in the current corrected image, extract and initialize the persistent micro-feature set, and construct a joint state vector containing the target motion, aberrations and feature three-dimensional coordinates.

[0064] Specifically, in the currently corrected thermal image, a rectangular region slightly larger than the predicted target contour from the previous period or the most recent high-precision anchored contour is defined as the region of interest. Within this region, an algorithm sensitive to low signal-to-noise ratio images is applied to detect local stable extrema of the radiation intensity distribution, such as phase-consistent singularities or cross-scale local maxima.

[0065] Points whose positional changes are less than a set threshold across multiple consecutive frames are selected to form an initial persistent micro-feature set, and their sub-pixel coordinates and relative radiometric intensities are recorded. Subsequently, a joint state vector is constructed, which unifies the target's motion state, the dynamic aberration state of the infrared window, and the estimated 3D coordinates of all micro-feature points in the target's body coordinate system into a single high-dimensional state. The initialization of this state vector incorporates the latest low-emissivity target motion and aberration information, and obtains the initial 3D coordinates of the micro-features through coarse back-projection. Its initial uncertainty is determined by the time span since the last high-precision anchoring.

[0066] 4b. In each processing cycle, the joint state vector is further forward-predicted based on the physical model to generate predictions for all observables in the next frame image.

[0067] Specifically, in each processing cycle, the target's motion state is predicted based on the target rigid body kinematics model and simplified aerodynamic constraints. The evolution of dynamic aberration states is predicted based on historical window aberration data or platform dynamics models. By combining the predicted target motion attitude and predicted aberration states, the three-dimensional coordinates of each persistent micro-feature point in the target body coordinate system are calculated. After coordinate transformation and perspective projection, its projected position on the ideal image plane is obtained. Using the aberration-to-geometric distortion mapping model established in Example 1, the image displacement caused by the predicted aberrations at this ideal projected position is calculated. The predicted coordinate positions of each micro-feature point in the next frame of the original distorted image are then synthesized.

[0068] 4c. After obtaining a new frame of corrected image, perform probabilistic correlation of weak signals near the predicted position, and use this soft observation to perform Bayesian update of the joint state vector.

[0069] 4d, when a natural transient event is detected, the combined state during gliding is strongly corrected and reset using high-precision observations.

[0070] Specifically, once the parallel-running Implementation Example 3 process detects a valid natural transient event and completes a high-precision measurement, it immediately uses this result to strongly correct the current joint state. The high-precision target contour, attitude, and sharp feature point coordinates obtained from the event measurement are used as quasi-true values ​​to directly reset the target motion state in the state vector. By comparing the observed positions of sharp feature points with the predicted positions based on the current aberration state, the window dynamic aberration state and its prediction model are corrected. Simultaneously, the 3D coordinate estimates of corresponding micro-feature points in the state vector are recalibrated or replaced with high-precision feature point coordinates, significantly reducing their uncertainty. Finally, the covariance matrix of the entire state estimate is reset to a small value determined by the event measurement accuracy, thereby eliminating the accumulated errors from long-term gliding tracking, restoring to a high-confidence state, and initiating a new tracking cycle.

[0071] The technical solutions described in the embodiments of this application above have at least the following technical effects or advantages: By jointly estimating target motion, window aberration, and persistent micro-features, autonomous, continuous, and robust tracking of low-observable targets is achieved in the absence of clear external events, significantly extending the maintenance time of high-precision sensing. Simultaneously, it can intelligently utilize any sudden natural events to reset accumulated errors during gliding, thereby providing stable, reliable target state information with explicit confidence assessments in any environment. This greatly enhances the ability to continuously monitor and strike concealed targets in complex adversarial environments.

[0072] Furthermore, embodiments of the present invention also provide an imaging distortion correction system for thermal imaging devices.

[0073] Figure 2 This is a schematic diagram of the structure of an imaging distortion correction method system for thermal imaging equipment according to an embodiment of the present invention.

[0074] like Figure 2 As shown, an imaging distortion correction method system for thermal imaging equipment includes: a data-driven module, a measurement and analysis module, an analysis and prediction module, a real-time calculation module, and an image processing module.

[0075] The data-driven module is used to train a quantitative mapping model from wavefront aberration coefficients to image geometric distortion field in a ground laboratory for thermal imaging devices equipped with irregularly curved infrared windows. This is achieved by actively injecting known infrared wavefront aberrations and simultaneously acquiring target thermal images.

[0076] The measurement and analysis module is used to record the motion trajectory of the device during operation. It uses the beam splitter to sense the wavefront distortion behind the infrared window in real time and calculates it into Zernike aberration coefficients that can be used to describe the distortion, serving as a real-time input for the cause of thermal image distortion.

[0077] The analysis and prediction module is used to fuse the real-time wavefront aberration coefficients of the device with historical aberration data to predict the wavefront aberration coefficients that will appear in the thermal imaging optical path at a fixed time in the future.

[0078] The real-time calculation module is used to input the predicted future aberration system into the quantitative mapping model, calculate the corresponding future thermal image distortion field, and combine it with the static distortion of the device to generate a dynamic pixel remapping lookup table for real-time image geometric correction.

[0079] The image processing module is used to perform pixel relocation and interpolation operations on the raw thermal image stream acquired in real time using a lookup table, and output a frame of thermal image that has been pre-compensated for future time correction.

[0080] It should be noted that other specific implementations of the imaging distortion correction method system for thermal imaging equipment according to the present invention can refer to the above-described imaging distortion correction method for thermal imaging equipment.

[0081] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0082] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0083] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0084] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0085] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0086] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0087] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. An imaging distortion correction method for a thermal imaging device, characterized by, The method includes: S1, for a thermal imaging device equipped with an irregularly shaped curved infrared window, a quantitative mapping model from wavefront aberration coefficients to image geometric distortion field is trained in a ground laboratory by actively injecting known infrared wavefront aberrations and simultaneously acquiring target thermal images; S2, the motion trajectory of the device during operation is recorded, and the wavefront distortion after passing through the infrared window is sensed in real time using a beam splitter and calculated into Zernike aberration coefficients that can be used to describe it, serving as the real-time causal input for the quantitative mapping model; S3, the real-time wavefront aberration coefficients of the device are fused with historical aberration data to predict the wavefront aberration coefficients that will appear in the thermal imaging optical path at a fixed time in the future; S4, the predicted future aberration coefficients are input into the quantitative mapping model to calculate the corresponding future thermal image distortion field, and combined with the static distortion of the device, a dynamic pixel remapping lookup table for real-time image geometric correction is generated; S5, the lookup table is used to perform pixel relocation and interpolation operations on the real-time acquired raw thermal image stream, and a frame of thermal image with pre-compensated future time correction is output.

2. The method of claim 1, wherein, The calculation of Zernike aberration coefficients that can be used to describe the wavefront aberration includes: splitting the incident infrared beam passing through the irregular curved window to obtain a beam for wavefront sensing; measuring the sensing beam using a wavefront sensor to obtain data reflecting wavefront distortion; and calculating and outputting coefficients that describe the current wavefront aberration based on the two measured data in real time.

3. The method of claim 1, wherein the method further comprises: The generation of a dynamic pixel remapping lookup table for real-time image geometric correction includes: calling a pre-calibrated static geometric distortion correction map; inputting the predicted future wavefront aberration information into a quantitative mapping model to obtain the corresponding dynamic geometric distortion data; fusing the static geometric distortion correction map with the dynamic geometric distortion data to generate a comprehensive correction map; determining the sub-pixel coordinates and interpolation weights of each pixel position in the corrected image in the original distorted image based on the comprehensive correction map; and generating a lookup table for image processing hardware to perform pixel remapping based on the corresponding coordinates and interpolation weights.

4. The method of claim 1, wherein, The output of a pre-compensated thermal image for future time correction includes: S51, after obtaining the corrected thermal image, when a potential low-emissivity target is identified in it, triggering a synchronous laser excitation process to acquire an excitation frame and a natural frame; S52, based on the continuity of the low-emissivity target's motion, mapping the clear features in the excitation frame to the natural frame for sub-pixel-level edge localization; S53, in the natural frame where edge localization has been completed, performing signal mixing modeling and calculating the coverage ratio for edge pixels; S54, fusing multiple sets of high-precision edge data and pixel mixing ratios to construct and update the enhanced geometric and radiation characteristic model of the low-emissivity target, and correcting the thermal image of the low-emissivity target in real time.

5. The method of claim 4, wherein the method further comprises: The triggering synchronous laser excitation process for acquiring excitation frames and natural frames includes: when the thermal imaging device is working, analyzing the obtained calibrated image to identify potential low-emissivity target areas; when the potential low-emissivity target area is identified, controlling the integrated pulsed laser to emit laser pulses towards the area, causing a brief thermal mark to form on the target surface; synchronously controlling the infrared detector to acquire images of the target after it has been heated during the laser pulse as excitation frames, and acquiring images of the target in its natural thermal state before and after the laser pulse as natural frames.

6. The method of claim 4, wherein the method further comprises: The triggering synchronous laser excitation process for acquiring excitation frames and natural frames also includes: S511, determining in real time whether naturally occurring infrared transient events have perceptual value for enhancing low emissivity targets based on the corrected thermal image; S512, when the transient event has perceptual value, acquiring a frame sequence covering the entire event process and extracting the intrinsic thermal features of the target caused by the transient event; S513, based on the extracted intrinsic thermal features of the target, quickly performing reverse calibration of the target's high-precision geometry and radiation characteristics to establish a golden period standard model for the current moment; S514, merging the golden period standard model into the quantitative mapping model to predict the precise contour and theoretical radiation pattern that the low emissivity target should present in the distortion-corrected image of the next frame.

7. The method of claim 6, wherein the method further comprises: The golden period standard model includes: a first process for instantaneous fine calibration of the target's geometry and spatial attitude, including extracting a clear outline of the target from its intrinsic thermal characteristics, registering it with historical predicted outlines, and analytically calculating the target's three-dimensional position, orientation, and outline dimensions in conjunction with the direction of the transient event source; and a second process for reverse calibration of the target's radiation characteristics, including measuring the radiation distribution on the target surface based on the target's intrinsic thermal characteristics, estimating its theoretical irradiance in conjunction with the physical model of the transient event source, and deducing the effective infrared emissivity distribution and basic radiation value of the target surface in reverse through physical relationships.

8. The method of claim 6, wherein the method further comprises: The method of predicting the precise contour of the low-emissivity target in the next frame of the distortion-corrected image includes: defining the target region of interest in the current corrected image, extracting and initializing a persistent set of micro-features, and constructing a joint state vector containing the target motion, aberrations, and feature three-dimensional coordinates; in each processing cycle, further forward prediction of the joint state vector based on the physical model to generate predictions for all observables in the next frame image; after obtaining the new frame of the corrected image, performing probabilistic correlation of weak signals near the predicted position, and using this soft observation to perform a Bayesian update of the joint state vector; when a natural transient event is detected, using high-precision observations to perform strong correction and reset of the joint state during gliding.

9. The method of claim 8, wherein the method further comprises: The generation of predictions for all observable quantities in the next frame image includes: predicting the target's motion state based on the target kinematic model and aerodynamic constraints; predicting the dynamic aberration state based on the window aberration evolution model; and calculating the predicted position of each persistent micro-feature in the next frame of the original distorted image by combining the predicted target motion state and aberration state.

10. An imaging distortion correction system for a thermal imaging device, applied to a method for imaging distortion correction of a thermal imaging device according to any one of claims 1 to 9, characterized in that, The system includes: The data-driven module is used to train a quantitative mapping model from wavefront aberration coefficients to image geometric distortion field in a ground laboratory for thermal imaging equipment equipped with irregularly curved infrared windows by actively injecting known infrared wavefront aberrations and simultaneously acquiring target thermal images. The measurement and analysis module is used to record the motion trajectory of the device during operation, and uses the beam splitter to sense the wavefront distortion after passing through the infrared window in real time, and calculates it into Zernike aberration coefficients that can be used to describe it, as a real-time input of the cause of thermal image distortion. The analysis and prediction module is used to fuse the real-time wavefront aberration coefficients of the device with historical aberration data to predict the wavefront aberration coefficients that will appear in the thermal imaging optical path at a fixed time in the future. The real-time calculation module is used to input the predicted future aberration system into the quantitative mapping model, calculate the corresponding future thermal image distortion field, and generate a dynamic pixel remapping lookup table for real-time image geometric correction by combining the static distortion of the device. The image processing module is used to perform pixel relocation and interpolation operations on the raw thermal image stream acquired in real time using a lookup table, and output a frame of thermal image that has been pre-compensated for future time correction.

Images