An adaptive time regularization method for UAV target tracking combined with Kalman filter

By introducing joint Kalman filtering and adaptive time regularization into the drone target tracking algorithm, the problems of low positioning accuracy and target drift are solved, and high-precision and real-time tracking are achieved in complex environments.

CN114677411BInactive Publication Date: 2024-11-29LIAONING TECHNICAL UNIVERSITY
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210363100.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-07
Publication Date
2024-11-29
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing drone target tracking algorithm has low positioning accuracy and is prone to target drifting under the conditions of messy background, change in scale, rapid movement, occlusion and rotation, and the fixed time regularization coefficient cannot adapt to the diversity of the target.

Method used

Adaptive time regularization method using combined Kalman filtering, the confidence of the target is calculated by the average peak correlation energy, adaptively adjust the time regularization parameters, and predict and correct the target position using Kalman filtering to improve tracking accuracy and robustness. sex.

Benefits of technology

It effectively improves the positioning accuracy and real-time nature of drone target tracking in complex environments, reduces the computational complexity, and enhances the adaptability and stability of the algorithm.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114677411B_ABST
    Figure CN114677411B_ABST
Patent Text Reader

Abstract

The present invention discloses an adaptive time regularization unmanned aerial vehicle target tracking method combined with Kalman filtering, including using average peak correlation energy to calculate the response score of the target position in the current frame, comparing the response score of the current frame with the response score of the initial frame to generate a confidence score, and judging the reliability of the target position; according to the obtained confidence score, adaptive adjustment of the time regularization parameter is realized to improve the adaptability of the algorithm; the unreliable target position is corrected through the predicted position of the Kalman filter to reduce the situation of target loss caused by large relative motion and motion blur interference, and improve the accuracy of target positioning. The adaptive time regularization unmanned aerial vehicle target tracking method combined with Kalman filtering of the present invention can effectively improve the positioning accuracy of the algorithm through the adaptive time regularization mechanism and Kalman filtering, and the low computational complexity can maintain real-time tracking of unmanned aerial vehicle targets.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of target tracking, and in particular relates to an adaptive time regularized unmanned aerial vehicle target tracking method combined with Kalman filtering. Background Art

[0002] Target tracking is one of the hottest research topics in the field of visual perception, and many applications have been generated by using the visual perception capabilities of drones, such as forest fire monitoring, aerial photography, and aerial refueling. However, drones are often disturbed by factors such as rapid motion, deformation, and motion blur during aerial tracking, resulting in reduced tracking performance. Therefore, it is of great significance to propose a robust and efficient drone target tracking algorithm.

[0003] At present, the tracking algorithm based on correlation filter has attracted extensive attention from scholars at home and abroad due to its high computational efficiency. Henriques et al. proposed the Kernelized Correlation Filters (KCF) visual tracking algorithm, which introduced the circular shift sampling method and multi-channel visual features to effectively optimize the filter coefficients in the frequency domain. Since the sampling method of KCF will cause boundary effects, the target search range is limited to a certain extent. When the target moves relatively large, tracking failure will occur. In response to this, Galoogahi et al. proposed the Background Aware Correlation Filters (BACF) visual tracking algorithm, which uses a binary matrix to crop the sample center area, effectively alleviating the boundary effect. However, in the case of complex backgrounds, more background noise will be introduced, making the filter unable to distinguish between real targets and similar targets. Later, in order to further improve the robustness of the algorithm, Fu et al. proposed the Boundary Effect-aware Visual Tracker (BEVT) algorithm based on BACF. The algorithm performs spatial penalties on the target according to its position, so that the algorithm can effectively suppress background noise while learning background information, thereby achieving better performance. Huang et al. proposed the Aberrance Repressed Correlation Filters (ARCF) algorithm, which limits the rate of change of the target response map. Although the above tracking algorithms have achieved satisfactory results in terms of accuracy and speed, the fixed regularization coefficients in the algorithm cannot adapt to most situations. When the target background is cluttered or deformed, the filter cannot fully learn the diverse information of the target appearance, which limits the tracking performance of the algorithm to a certain extent. Therefore, Dai et al. introduced adaptive spatial regularization constraints based on SRDCF and proposed an adaptive spatial regularized correlation filter (ASRCF) algorithm, which adaptively updates the spatial weight of each frame according to the appearance changes of the sample, and assigns different weights to the noise part and reliable part of the sample, thereby effectively improving the adaptability of the filter.

[0004] At present, among the target tracking algorithms based on correlation filtering, the STRCF algorithm has achieved good tracking performance. The algorithm introduces a time regularization term based on the SRDCF algorithm, effectively avoiding the degradation of the filter over time and improving the robustness of the algorithm. Then, the filter coefficients are solved by the alternating direction multiplier method to ensure the computational efficiency, so that the algorithm can perform real-time tracking at a speed 5 times that of the SRDCF algorithm. It has achieved better performance than the SRDCF algorithm in terms of accuracy and speed.

[0005] The STRCF algorithm uses the temporal regularization term to keep the filters between adjacent frames as similar as possible, which can avoid the target drift problem caused by target deformation and partial occlusion, and solve the filter coefficients by the alternating direction multiplication method to improve the tracking speed. However, when the filter estimates the target state, due to the cluttered background, scale changes, rapid motion, occlusion and rotation of the target, a certain degree of process noise will be introduced, and the error will make the target state estimation inaccurate, and the fixed temporal regularization coefficient cannot adapt to most situations, which will cause the filter to be unable to fully learn the reliable information of the target, resulting in target drift. Summary of the invention

[0006] In view of the shortcomings of the prior art, the problem to be solved by the present invention is to provide an adaptive time regularization UAV target tracking method combined with Kalman filtering, which improves the UAV target tracking performance of STRCF under conditions of background clutter, scale change, rapid motion, occlusion and rotation, effectively improves the positioning accuracy, and can maintain good real-time performance.

[0007] In order to solve the above technical problems, the present invention is implemented by the following technical solutions:

[0008] The method for tracking unmanned aerial vehicle targets using adaptive time regularization combined with Kalman filtering provided by the present invention comprises the following steps:

[0009] S1: Calculate the response score of the target position in the current frame using the average peak correlation energy, compare the response score of the current frame with the response score of the initial frame to generate a confidence score to determine the reliability of the target position;

[0010] S2: Adaptive adjustment of the temporal regularization parameter is achieved according to the obtained confidence score to improve the adaptability of the algorithm;

[0011] S3: Correct the unreliable target position through the predicted position of Kalman filtering to reduce the target loss caused by large relative motion and motion blur interference, and improve the accuracy of target positioning.

[0012] Furthermore, in step S2, the confidence of the target in the current frame is calculated by APCE, and APCE can be expressed as:

[0013]

[0014] Where F max and F min are the maximum and minimum values ​​of the target response, respectively, and F m,n is the response value at position (m,n).

[0015] Furthermore, in step S3, the motion model of the Kalman filter can be expressed as:

[0016] X t =FX t-1

[0017] Z t =HX t +V t

[0018] Where F is the state transfer matrix, H is the measurement matrix, V t is the measurement noise, Z t is the observation at the tth frame.

[0019] From the above, the present invention has the following beneficial effects:

[0020] 1. The present invention introduces adaptive time regularization term and Kalman filter into STRCF algorithm, uses adaptive time regularization term to realize adaptive adjustment of time penalty coefficient, reduces time penalty coefficient in the area with high target reliability, makes filter learn the appearance change of target more fully, can track UAV target more stably, when target reliability is low, keeps initial time penalty coefficient, predicts target position by Kalman filter, uses prediction value to correct filter estimation value, makes target positioning more accurate.

[0021] 2. Based on the APCE adaptive time regularization mechanism and the implementation of Kalman filtering, the UAV target position is corrected when there is interference in the target area. The adaptive time regularization mechanism and Kalman filtering can effectively improve the positioning accuracy of the algorithm, and the low computational complexity can maintain real-time tracking of the UAV target.

[0022] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following is a detailed description in combination with the preferred embodiments and the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the drawings of the embodiment are briefly introduced below.

[0024] Figure 1 The present invention is a flow chart of the adaptive time regularization UAV target tracking method combined with Kalman filtering. DETAILED DESCRIPTION

[0025] like Figure 1 As shown, the adaptive time regularization UAV target tracking method of the joint Kalman filter of the present invention uses the average peak correlation energy to calculate the response score of the target position in the current frame, compares the response score of the current frame with the response score of the initial frame to generate a confidence score, and judges the reliability of the target position. Then, according to the obtained confidence score, the time regularization parameter is adaptively adjusted to improve the adaptability of the algorithm. Finally, the unreliable target position is corrected by the predicted position of the Kalman filter to reduce the loss of the target due to interference such as large relative motion and motion blur, and improve the accuracy of target positioning.

[0026] The STRCF algorithm estimates the position and scale of the target through a correlation filter. The correlation filter coefficient f can be solved by the following objective function:

[0027]

[0028] Where · is the dot product symbol and * is the correlation operator symbol. x is the target sample extracted at the tth frame and y is the predefined Gaussian response. d is the feature channel index and d = {1, 2, 3, .., D}, D is the total number of feature channels. The second term is the spatial regularization constraint, w represents the spatial penalty coefficient. The third term is the temporal regularization constraint, μ is the temporal penalty coefficient.

[0029] In order to improve the computational efficiency, an auxiliary variable g and a step size parameter γ are introduced, where g = f. Then, the alternating direction multiplier method is used to solve the above objective function. The augmented Lagrangian form of equation (1) is:

[0030] Where s is the Lagrange multiplier, and h = s / γ.

[0031] Using ADMM, equation (2) is transformed into two easily solvable sub-problems:

[0032]

[0033] Where i represents the number of iterations, and i=2.

[0034] Finally, the target response function of the current frame is obtained by operating the correlation filter f with the feature samples extracted from the target candidate area. The maximum response value corresponds to the position and scale of the target in the current frame. The target response function is:

[0035]

[0036] Where ^ is the Fourier transform of the variable.

[0037] In the update phase of the filter, in order to make it more fully learn the appearance changes of the target with high reliability, the present invention introduces an adaptive temporal regularization mechanism based on APCE, and calculates the confidence of the target in the current frame through APCE. APCE can be expressed as:

[0038]

[0039] Where F max and F min are the maximum and minimum values ​​of the target response, respectively, and F m,n is the response value at position (m,n).

[0040] Then, the temporal penalty coefficient is adaptively adjusted through formula (6). When the reliability is high, the temporal penalty coefficient is reduced so that the filter can learn the changes of the target appearance model more fully.

[0041]

[0042] Chinese APCE t is the response confidence of the tth frame, APCE0 is the confidence at the initial frame, μ0 is the initial time penalty coefficient, tr0 is the threshold for determining whether the tracking result is reliable, and tr0 = 0.7.

[0043] In order to further improve the accuracy of the algorithm, after judging the reliability through formula (6), when the reliability is low, the target position of the UAV is corrected by the predicted value of the Kalman filter. The motion model of the Kalman filter can be expressed as:

[0044] X t =FX t-1 (7)

[0045] Z t =HX t +V t (8)

[0046] Where F is the state transfer matrix, H is the measurement matrix, V t is the measurement noise, Z t is the observation at the tth frame.

[0047] The state estimation process of Kalman filtering is divided into a prediction step and an update step, where the prediction process can be performed by the following formula:

[0048]

[0049] P t / -1 =FP t-1 F T (10)

[0050] In the formula is the optimal estimate of the state of the previous frame, P t-1 is the error covariance matrix of the optimal state estimate for the previous frame.

[0051] Then, the Kalman gain Kg is calculated by equation (11):

[0052] Kg=P t / t-1 H T (HP t / t-1 H T +R) (11)

[0053] Where R is the measurement error covariance matrix.

[0054] Finally, the optimal state estimate of the current frame is calculated by equation (12): And use formula (13) to update the covariance matrix P t :

[0055]

[0056] P k =(I-Kg*H)P k / k-1 (13)

[0057] Where I is the unit matrix.

[0058] The present invention introduces an adaptive time regularization term and a Kalman filter into the STRCF algorithm, uses the adaptive time regularization term to realize the adaptive adjustment of the time penalty coefficient, reduces the time penalty coefficient in the area with high target reliability, enables the filter to learn the appearance change of the target more fully, and can track the UAV target more stably; when the target reliability is low, maintains the initial time penalty coefficient, predicts the target position through the Kalman filter, and uses the predicted value to correct the estimated value of the filter, so that the target positioning is more accurate.

[0059] The above is only a preferred embodiment of the present invention, which certainly cannot be used to limit the scope of rights of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and changes can be made without departing from the principle of the present invention, and these improvements and changes are also regarded as the protection scope of the present invention.

Claims

1. An adaptive time regularization UAV target tracking method combined with Kalman filtering, It is characterized in that The following steps are involved: S1: Calculate the response score of the target position in the current frame using the average peak correlation energy, compare the response score of the current frame with the response score of the initial frame to generate a confidence score to determine the reliability of the target position; S2: Adaptive adjustment of the temporal regularization parameter is achieved according to the obtained confidence score to improve the adaptability of the algorithm; S3: Correct the unreliable target position through the predicted position of Kalman filter to reduce the target loss caused by large relative motion and motion blur interference, and improve the accuracy of target positioning; In step S2, the confidence of the target in the current frame is calculated by APCE, which can be expressed as: Where F max and F min are the maximum and minimum values ​​of the target response, respectively, and F m,n is the response value at position (m,n); The time penalty coefficient is adaptively adjusted by the following formula. When the reliability is high, the time penalty coefficient is reduced so that the filter can learn the changes of the target appearance model more fully. m t =μ 0 ,r t <tr 0 m t =μ 0 ·(1.6-r t ),otherwise Chinese APCE t is the response confidence of the tth frame, APCE 0 is the confidence at the initial frame, μ 0 is the initial time penalty coefficient, tr 0 is the threshold that determines whether the tracking result is reliable, and tr 0 =0.

7.

2. The method for tracking unmanned aerial vehicle targets using adaptive time regularization combined with Kalman filtering as claimed in claim 1, It is characterized in that In step S3, the motion model of Kalman filtering can be expressed as: X t =FX t-1 Z t =HX t +V t Where F is the state transfer matrix, H is the measurement matrix, V t is the measurement noise, Z t is the observation at the tth frame.

Citation Information

Patent Citations

  • KFSTRCF-based target tracking architecture

    CN111402303A

  • Self-adaptive context-aware anti-occlusion target tracking method and device

    CN113850847A