Kalman filtering infrared target tracking method based on temperature state expansion

Abstract

The present application relates to a kind of infrared target tracking method based on temperature state extension kalman filtering, belong to infrared image processing and target tracking technical field, by temperature and temperature change rate is extended as extended kalman filtering state variable, and temperature dynamic prior equation describing heat accumulation and cooling process is established as state transition physical constraint, so that temperature can be shielded or in motion blur causes the ability of independent maintenance temperature and position joint state estimation when light flow feature is degraded;Infrared image is input into YOLOv8 network for target detection, introduce the dynamic adjustment mechanism of adaptive observation noise covariance based on YOLOv8 detection confidence, realizes the automatic matching of detection measurement reliability and filter update weight;Establish extended kalman filtering model, temperature and temperature change rate are included in filter framework as independent state variable;And introduce temperature feature auxiliary trajectory-target matching decision, effectively reduce the ID switching rate in multi-target thermal radiation scene.

Description

Technical Field

[0001] This invention relates to the field of infrared image processing and target tracking technology, specifically to a Kalman filter-based infrared target tracking method based on temperature state extension. Background Technology

[0002] Infrared imaging technology, with its passive detection, all-weather operation, and strong anti-interference capabilities, possesses irreplaceable advantages in target tracking. Especially at night, in low-visibility conditions, or under complex electromagnetic environments, infrared systems can effectively capture differences in thermal radiation from object surfaces, enabling reliable detection and tracking of moving targets. Compared to traditional cooled detectors, uncooled infrared focal plane array detectors offer significant advantages such as no need for cryogenic cooling, low power consumption, small size, low cost, and fast response speed, making them the preferred solution for miniaturized and integrated infrared devices. However, uncooled infrared detectors, due to their relatively long integration time, are prone to motion blur when the target is moving at high speed, the carrier platform is shaking, or the line of sight changes. This leads to decreased image contrast, target edge diffusion, and blurred thermal features, severely impacting subsequent detection and tracking performance. Furthermore, the temperature signal in infrared images, a core feature for target identification, is susceptible to interference from various factors: detector fixed pattern noise, non-uniform response, environmental radiation reflection, atmospheric transmission attenuation, and uncertainty in target surface emissivity. This results in significant fluctuations in temperature observations, with static measurement errors often exceeding ±5°C.

[0003] Existing infrared target tracking methods based on optical flow and Kalman filtering suffer from the following fundamental drawbacks: their Kalman filter state vectors only contain target position and velocity, completely neglecting temperature modeling; more importantly, Kalman filtering is only passively activated as a position prediction remedy after the target is deemed lost, and does not participate in state estimation during normal tracking. Similarly, twin network tracking methods incorporating Kalman filtering employ a confidence threshold binary switching strategy, only activating Kalman prediction when the confidence level is below the threshold, lacking a continuous and smooth adjustment mechanism for observation update weights. When motion blur leads to degradation of detection features, these methods, lacking an auxiliary tracking mechanism based on thermal features, result in a significantly increased tracking interruption rate in high-speed motion scenarios.

[0004] Another type of method that incorporates temperature features into infrared target recognition, while improving the accuracy of distinguishing targets from background interference by utilizing temperature time series, essentially only uses temperature as a discriminative feature input for time series classifiers such as LSTM, without modeling temperature as an independent state variable in the state estimation framework. This type of method solves the problem of identifying and classifying targets and backgrounds in a single frame or short sequence, rather than the problem of continuous multi-frame trajectory tracking: temperature information cannot be filtered to smooth noise, nor can it be used to independently maintain tracking through dynamic temperature priors when the target is occluded; at the same time, temperature features do not participate in data association decisions, and when multiple thermal radiation targets exist simultaneously, association algorithms that rely solely on position features are prone to trajectory ID switching.

[0005] The aforementioned problems are particularly prominent in low-cost infrared tracking systems: low tracking success rate, unstable temperature measurement, and insufficient overall system robustness, limiting the application potential of uncooled infrared devices in complex dynamic scenarios. Therefore, researching a tracking method that can fully utilize temperature dynamic characteristics, suppress noise and ambiguity interference, and achieve joint position and temperature estimation is of great significance for improving the performance of miniaturized infrared systems, reducing deployment costs, and expanding application scope. Summary of the Invention

[0006] The purpose of this invention is to provide a Kalman filter infrared target tracking method based on temperature state extension to solve the problems existing in the background art.

[0007] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: a Kalman filter infrared target tracking method based on temperature state extension, comprising the following steps: A temperature observation model is established. Based on the gray-level-temperature mapping relationship established in advance by radiometric calibration, dynamic temperature observation values ​​are extracted from the gray-level distribution of the target image. A temperature dynamic prior equation is established to incorporate the physical processes of target heat accumulation and cooling into the extended Kalman filter state transition constraints. Infrared images are input into the YOLOv8 network for target detection, and real-time temperature observations are extracted within the detection box based on the temperature observation model. The position and temperature are combined to form an observation vector. The observation noise covariance matrix is ​​adaptively set based on the confidence level of YOLOv8 output; An extended Kalman filter model is established, incorporating temperature and the rate of temperature change as independent state variables into the filtering framework. This model is then combined with the observed vectors for state prediction, observation updates, and covariance correction. In the data association stage, the location-temperature joint data association cost matrix is ​​constructed by weighted fusion of Mahalanobis distance and temperature residuals, where the weights are adaptively adjusted based on confidence. Temperature features are introduced to assist in trajectory-target matching decisions, and the optimal allocation is solved using the Hungarian algorithm.

[0008] Preferably, the extracted dynamic temperature observations include hotspot temperatures calculated within the target detection bounding box. and regional average temperature .

[0009] Preferably, the temperature dynamic prior equation is: ; ; in, denoted as the thermal decay coefficient, k as the discrete time step index, T as the target temperature state variable, t as the time variable, dt as the time interval between adjacent frames, and dT / dt as the temperature change rate.

[0010] Preferably, the observation vector is: .

[0011] Preferably, an adaptive mechanism is introduced to adaptively set the observation noise covariance matrix. The adaptive mechanism is as follows: when the confidence level is greater than or equal to a preset threshold, the observation noise covariance matrix equals... The filter fully trusts the measurement; when the confidence level is less than the preset threshold, the observation noise covariance matrix increases as the confidence level decreases, the filter reduces the measurement update gain and tends to depend on the temperature dynamic prior equation and motion model.

[0012] ; in, For confidence level, For the threshold, To minimize the observation noise covariance matrix, This is the covariance adjustment factor.

[0013] Preferably, the state vector of the extended Kalman filter model is: where x and y are positions. Let T be the velocity and T be the target temperature state variable. This represents the rate of temperature change.

[0014] Preferably, the location-temperature joint data association cost matrix is: ; in, The Mahalanobis distance between the predicted position of trajectory i and the detection j. For the temperature state of trajectory i predicted by EKF, To detect the real-time temperature of j; Indicates weight, and , .

[0015] Preferably, the temperature feature-assisted trajectory-target matching decision is as follows: when the position predictions of multiple trajectories fall within the association threshold of the same detection target at the same time, and the Mahalanobis distance alone cannot reliably distinguish them, the temperature residual is used as an independent discrimination dimension to participate in the cost matrix calculation, so that the cost of the correct match is significantly less than the cost of the incorrect match.

[0016] Preferably, the predicted bounding box position output by the EKF prediction stage in the extended Kalman filter model is fed back to the YOLOv8 network to define the region of interest for YOLOv8 object detection in the current frame.

[0017] Preferably, in the case of occlusion or failed detection frames, only state prediction is performed without measurement updates. The temperature dimension is continuously extrapolated based on the temperature dynamic prior equation, and the position dimension is continuously predicted based on the constant velocity motion model; the candidate trajectory is continuously... After successful frame association, it is upgraded to a formal trajectory; the number of consecutive lost frames exceeds [number missing]. The trajectory terminates at that time.

[0018] Beneficial effects: This invention extends temperature and temperature change rate to extended Kalman filter state variables, and establishes a temperature dynamic prior equation describing the heat accumulation and cooling process as a physical constraint for state transition, enabling the temperature to be masked or to independently maintain the joint state estimation of temperature and position when motion blur causes optical flow features to degrade. This invention incorporates temperature as a state variable into the Kalman filter framework for smooth estimation, and significantly reduces the temperature measurement error from ±5°C or more of the original observation through a filtering noise suppression mechanism. At the same time, temperature features participate in the decision of the location-temperature joint data association cost matrix, effectively reducing the trajectory ID switching rate in multi-target thermal radiation scenarios. It fundamentally solves the problem of continuous target trajectory tracking, rather than just identifying and classifying targets and backgrounds, and there are essential differences in functional positioning and technical approach.

[0019] This invention introduces an adaptive observation noise covariance dynamic adjustment mechanism based on YOLOv8 detection confidence, realizing automatic matching between detection measurement reliability and filter update weights: when the detection is clear, real-time observation is maximized; when the detection is blurry, the observation noise covariance automatically increases as the confidence decreases, and the filter shifts to rely on temperature dynamic priors and motion models. Tracking stability can be maintained without additional deblurring preprocessing, making it suitable for real-time deployment on resource-constrained embedded platforms (such as ARM / DSP). Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the Kalman filter infrared target tracking method based on temperature state extension according to the present invention. Detailed Implementation

[0021] To make the objectives and advantages of this invention clearer, the invention will be specifically described below with reference to embodiments. It should be understood that the following text is merely used to describe one or more specific embodiments of the invention and does not strictly limit the scope of protection specifically claimed by the invention.

[0022] Example: A Kalman filter-based infrared target tracking method based on temperature state extension, such as... Figure 1 As shown, it includes the following steps: Step 1: Establish an infrared image sequence dataset. Use an uncooled strapdown infrared detector and a controllable turntable to capture infrared image datasets with multiple levels of ambiguity. The targets are mainly rotary-wing UAVs, fixed-wing UAVs, unmanned surface vessels, or other moving targets with significant thermal radiation. The infrared image dataset includes sequence images with different speeds, attitudes, and background interference. In this embodiment, an uncooled infrared thermal imager is mounted on a precision turntable. Different jitter amplitudes (0-15°) and frequencies (1-5Hz) are set through the turntable control system. Based on complex backgrounds such as the sky, buildings, and vegetation, and with rotary-wing drones or vehicles as the main targets, infrared image sequences covering various motion speeds, attitude changes, and interference conditions are captured. The dataset includes samples with low contrast, partial occlusion, and different degrees of blur. The dataset is divided into training and testing sets in a 4:1 ratio to ensure the diversity of model training and the objectivity of evaluation.

[0023] Step 2: Construct a temperature observation model, analyze the correspondence between infrared image grayscale values ​​and detector-received radiation intensity, and convert the grayscale distribution into temperature observation values ​​using calibration curves or a simplified Planck radiation formula, thereby establishing a mapping relationship. This considers target surface emissivity correction, atmospheric transmission path attenuation compensation, and the radiation energy diffusion effect across pixel motion blur caused by the long integration time of uncooled detectors, as well as the effects of emissivity correction, atmospheric transmission attenuation, and environmental reflection. In this embodiment, dynamic temperature observation values ​​are extracted from the target's image grayscale distribution, and a dynamic prior temperature equation is established. This equation incorporates the physical processes of target heat accumulation and cooling into the extended Kalman filter state transition constraints. Specifically: Calculate hotspot temperature within the target detection bounding box and regional average temperature Emissivity correction and atmospheric attenuation compensation are introduced to establish a Gaussian distribution model of temperature observation noise, providing reliable input for subsequent filtering. Simultaneously, a dynamic temperature prior is defined, including models with linear rates of change or exponential approaches to ambient temperature, to describe the target's heat accumulation or cooling process. The specific dynamic temperature prior equation is as follows: ; ; in, The thermal decay coefficient is taken as the thermal decay coefficient during the continuous operation of the target engine and the heat accumulation stage. (Temperature rises at approximately a constant rate), during the target cooling phase, take... (The temperature index decays and approaches the ambient temperature). The values ​​can be preset according to the target type (rotor / fixed wing / vehicle, etc.) and historical thermal characteristics statistics. k is the discrete time step index (i.e. the kth frame), T is the target temperature state variable, t is the time variable, dt is the time interval between adjacent frames (sampling period), and dT / dt is the temperature change rate (i.e., the first derivative of temperature with respect to time).

[0024] This temperature dynamic prior equation incorporates the physical processes of target heat accumulation and cooling into the extended Kalman filter state transition constraints, giving the filter the ability to independently predict the temperature state in target occlusion frames, which is different from existing methods that only use temperature time series information as the discriminative feature input of time series classifiers.

[0025] Step 3: Input the infrared image sequence into a pre-trained and fine-tuned YOLOv8 network, using a lightweight YOLOv8n to ensure real-time performance. The network outputs target bounding boxes, confidence scores, and category information. Simultaneously, the temperature observation model described in Step 2 is applied within each detection box to calculate real-time temperature observations, including hotspot temperatures, average temperatures, and their statistical variance, which serve as the observation input for the filter. This step effectively handles low-contrast and blurred images, providing stable joint location and temperature observations. In this embodiment, location and temperature are combined into a joint observation vector: ; In this embodiment, to avoid passively activating Kalman filtering for position prediction only after the target is determined to be occluded or lost, an adaptive mechanism is introduced to adaptively set the observation noise covariance matrix. The adaptive mechanism is as follows: ; in, For confidence level, For the threshold, To minimize the observation noise covariance matrix, This is the covariance adjustment factor; where Based on the target type preset (smaller value for thermally stable targets). Rapidly changing targets );when hour The filter fully trusts the measurement; when hour, The filter automatically increases as confidence decreases, while the measurement update gain (Kalman gain K decreases) decreases. It relies more on dynamic temperature priors and motion model state predictions to maintain tracking stability in frames where motion blur reduces detection confidence. This adaptive approach requires no deblurring preprocessing. The mechanism enables the filter to actively participate in the tracking process throughout, and the observation update weights are continuously and smoothly adjusted according to the detection quality.

[0026] Step 4: Establish an extended Kalman filter model, extending the state vector to: ; Where x and y are positions. Let T be the velocity and T be the target temperature state variable. The temperature change rate is represented by the first four dimensions, which are traditional kinematic parameters, and the last two dimensions, which are extended temperature dimensions. The temperature T and the temperature change rate are then considered together. The state variables are incorporated into the filtering framework as independent state variables; a linear or nonlinear dynamic model is adopted (EKF is used when the temperature is nonlinear), and the position and temperature observations in step S3 are used for state prediction, observation update and covariance correction; among them, a constant velocity motion model is used in combination with the temperature linear or nonlinear dynamic equation (EKF linearization is switched when the temperature changes drastically); the tracking process for each frame is divided into the following four stages: (1) State prediction stage: Using the state transition matrix F and process noise covariance Q, forward time inference is performed on the state vector and covariance matrix of all confirmed trajectories to obtain the prior state estimate and predicted bounding box position of the current frame; the predicted bounding box position is simultaneously fed back to the YOLOv8 detection stage in step 3 to limit the detection of the region of interest (ROI) in the current frame and construct the detection-tracking closed-loop collaboration. (2) Data association stage: Match and associate the target observations detected by YOLOv8 with the predicted states of existing trajectories to construct a joint location-temperature cost matrix: ; in, The Mahalanobis distance between the predicted position of trajectory i and the detection j. For the temperature state of trajectory i predicted by EKF, To detect the real-time temperature of j; Indicates weight, and , The system adaptively adjusts with YOLOv8 confidence; when the detection is reliable, location features dominate, and when the target is ambiguous, thermal features assist in association; the Hungarian algorithm is used to solve for the global optimal allocation, and three types of results are obtained: matching pairs, unmatched detections, and unmatched trajectories. Introducing temperature features to assist trajectory-target matching decision-making: When the position predictions of multiple trajectories fall within the association threshold of the same detected target, and Mahalanobis distance alone cannot reliably distinguish them, the temperature residual is used as an independent discrimination dimension to participate in the cost matrix calculation, so that the cost of the correct match with similar thermal radiation characteristics is significantly less than the cost of the incorrect match; thereby effectively reducing the ID switching rate in multi-target thermal radiation scenarios. The Hungarian algorithm is used to solve the global optimal one-to-one correspondence allocation based on the above location-temperature joint cost matrix. The allocation results include: (i) matching pairs, which serve as the observation input for the Kalman gain update in the subsequent (3); (ii) unmatched detection, which serve as the input for the initialization of new candidate trajectories; (iii) unmatched trajectories, which enter the prediction maintenance mode or trajectory termination determination process; thereby realizing a reliable association between the detected target and the existing tracking trajectory. (3) State update and trajectory management stage: For successfully matched trajectories, the bounding box center coordinates and temperature observations are fused, and the state vector and covariance matrix are corrected by Kalman gain K to complete the filtering update; for unmatched detections, they are determined to be potential new targets, new candidate trajectories are initialized, and they are confirmed as official trajectories after successful association in multiple consecutive frames; for unmatched existing trajectories, in frames where detection fails or is severely blurred, only state prediction is performed without measurement update, the temperature dimension is continuously extrapolated based on the temperature dynamic prior equation, and the position dimension is continuously predicted based on the constant velocity motion model; the candidate trajectories are continuously updated in the following frames. After successful frame association, it is upgraded to a formal trajectory; the number of consecutive lost frames exceeds [number missing]. The trajectory is terminated at a certain time, enabling dynamic creation, maintenance, and deletion of trajectories; (4) Trajectory output stage: Integrate the filtered states of all active trajectories, output a smooth target trajectory sequence, predicted bounding box coordinates and size, dynamic temperature curve and trajectory confidence score, and support trajectory backtracking and historical state query, providing complete tracking results for subsequent high-level applications; This multi-stage framework effectively solves the tracking continuity problem in complex scenarios such as target appearance, disappearance, occlusion and intersection, and significantly improves the anti-interference ability and tracking robustness of the system.

[0027] Based on the detailed description above, this invention provides an infrared target tracking method based on temperature state extension. This method uses temperature dynamics as the core constraint, achieves efficient detection through the YOLOv8 network, extracts reliable features using a temperature observation model, and establishes an extended Kalman filter to jointly estimate position and temperature, thereby achieving stable tracking and dynamic temperature measurement under noise suppression and ambiguity interference.

[0028] The invention described above can also be extended to other similar infrared tracking tasks, such as personnel monitoring, vehicle following, or industrial thermal fault detection. Ideal performance can be obtained simply by adjusting the emissivity parameters, dynamic model, or fine-tuning YOLOv8 and filter initial values ​​on a similar dataset.

[0029] The embodiments of the present invention have been described in detail above with reference to the examples. However, the present invention is not limited to the above embodiments. For those skilled in the art, after learning the contents described in the present invention, several equivalent changes and substitutions can be made without departing from the principle of the present invention. These equivalent changes and substitutions should also be considered to fall within the protection scope of the present invention.

Claims

1. A Kalman filter-based infrared target tracking method based on temperature state extension, characterized in that: Includes the following steps: A temperature observation model is established. Based on the gray-level-temperature mapping relationship established in advance by radiometric calibration, dynamic temperature observation values ​​are extracted from the gray-level distribution of the target image. A temperature dynamic prior equation is established to incorporate the physical processes of target heat accumulation and cooling into the extended Kalman filter state transition constraints. Infrared images are input into the YOLOv8 network for target detection, and real-time temperature observations are extracted within the detection box based on the temperature observation model. The position and temperature are combined to form an observation vector. The observation noise covariance matrix is ​​adaptively set based on the confidence level of YOLOv8 output; An extended Kalman filter model is established, incorporating temperature and the rate of temperature change as independent state variables into the filtering framework. This model is then combined with the observed vectors for state prediction, observation updates, and covariance correction. In the data association stage, the location-temperature joint data association cost matrix is ​​constructed by weighted fusion of Mahalanobis distance and temperature residuals, where the weights are adaptively adjusted based on confidence. Temperature features are introduced to assist in trajectory-target matching decisions. The optimal allocation is solved using the Hungarian algorithm. The temperature dynamic prior equation is: ； ； in, denoted as the thermal decay coefficient, k as the discrete time step index, T as the target temperature state variable, t as the time variable, dt as the time interval between adjacent frames, and dT / dt as the temperature change rate.

2. The Kalman filter infrared target tracking method based on temperature state extension according to claim 1, characterized in that: The extracted dynamic temperature observations include hotspot temperatures calculated within the target detection bounding box. and regional average temperature .

3. The Kalman filter infrared target tracking method based on temperature state extension according to claim 2, characterized in that: The observation vector is: .

4. The Kalman filter infrared target tracking method based on temperature state extension according to claim 1, characterized in that: An adaptive mechanism is introduced to adaptively set the observation noise covariance matrix. The adaptive mechanism is as follows: when the confidence level is greater than or equal to a preset threshold, the observation noise covariance matrix equals... The filter fully trusts the measurement; when the confidence level is less than the preset threshold, the observation noise covariance matrix increases as the confidence level decreases, the filter reduces the measurement update gain and tends to depend on the temperature dynamic prior equation and motion model; ； in, For confidence level, For the threshold, To minimize the observation noise covariance matrix, This is the covariance adjustment factor.

5. The Kalman filter infrared target tracking method based on temperature state extension according to claim 4, characterized in that: The state vector of the extended Kalman filter model is: where x and y are positions. Let T be the velocity and T be the target temperature state variable. This represents the rate of temperature change.

6. The Kalman filter infrared target tracking method based on temperature state extension according to claim 5, characterized in that: The location-temperature joint data association cost matrix is ​​as follows: ； in, The Mahalanobis distance between the predicted position of trajectory i and the detection j. For the temperature state of trajectory i predicted by EKF, To detect the real-time temperature of j; Indicates weight, and , .

7. The Kalman filter infrared target tracking method based on temperature state extension according to claim 6, characterized in that: The temperature feature-assisted trajectory-target matching decision is as follows: when the position predictions of multiple trajectories fall within the association threshold of the same detection target, and Mahalanobis distance alone cannot reliably distinguish them, the temperature residual is used as an independent discrimination dimension to participate in the cost matrix calculation, so that the cost of the correct match is significantly less than the cost of the incorrect match.

8. The Kalman filter infrared target tracking method based on temperature state extension according to claim 5, characterized in that: The predicted bounding box position output from the EKF prediction stage in the extended Kalman filter model is fed back to the YOLOv8 network to define the region of interest for YOLOv8 object detection in the current frame.

9. A Kalman filter infrared target tracking method based on temperature state extension according to any one of claims 1-8, characterized in that: In the case of occlusion or failed detection frames, only state prediction is performed without measurement updates. The temperature dimension is continuously extrapolated based on the aforementioned temperature dynamic prior equation, and the position dimension is continuously predicted based on a constant velocity motion model. Candidate trajectories are continuously... After successful frame association, it is upgraded to a formal trajectory; the number of consecutive lost frames exceeds [number missing]. The trajectory terminates at that time.

Images