A UAV target tracking method and system based on adaptive correlation filter

Abstract

This application discloses a method and system for UAV target tracking based on an adaptive correlation filter. The method includes: acquiring the original features of video frames and performing rotation correction processing to obtain several video frame rotation correction features; constructing adaptive feature residual weights and adaptive regularization coefficients based on the several video frame rotation correction features; introducing adaptive feature residual weights and adaptive regularization coefficients based on the correlation filter and optimizing them through the ADMM algorithm to construct an adaptive correlation filter; and convolving the adaptive correlation filter with the original features of the video frames to obtain the position of the UAV target being tracked. The embodiments of this application can enhance the adaptability and generalization ability of the tracker, adapt to complex and changing environments, and improve the tracking accuracy of UAVs. This application can be widely applied in the field of UAV target tracking technology.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) target tracking technology, and in particular to a UAV target tracking method and system based on an adaptive correlation filter. Background Technology

[0002] As an important branch of computer vision, target tracking technology's core task is to accurately locate and track the position and dynamic information of specific targets in a scene through the analysis of continuous image sequences. Early target tracking research mainly focused on generative algorithms, but these algorithms struggled to meet the needs of practical applications. This led to the development of discriminative algorithms, which directly infer the target's position by comparing the differences between the target model and the background, greatly improving efficiency. Researchers have introduced machine learning methods such as structured learning and random forests to train models, achieving significant results. However, existing target tracking technologies still have some shortcomings, such as difficulty in recognizing targets under rotational changes, or the fact that the robustness of the tracker is often improved by directly utilizing background features or historical filtering, meaning the filters do not fully consider the consistency of background information. Furthermore, fixed learning rates perform poorly in dealing with the dynamic influence of historical information. Therefore, existing target tracking technologies cannot adapt to complex and changing environments, and their tracking accuracy needs improvement.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0004] The main objective of this application is to propose a UAV target tracking method and system based on an adaptive correlation filter, which can enhance the tracker's adaptability and generalization ability, adapt to complex and ever-changing environments, and improve UAV tracking accuracy.

[0005] To achieve the above objectives, one aspect of this application proposes a UAV target tracking method based on an adaptive correlation filter, the method comprising:

[0006] The original features of the video frames are obtained and rotation correction is performed to obtain several video frame rotation correction features.

[0007] Based on several video frame rotation correction features, adaptive feature residual weights and adaptive regularization coefficients are constructed;

[0008] Based on the correlation filter, the adaptive feature residual weights and the adaptive regularization coefficients are introduced, and the adaptive correlation filter is constructed by optimizing it through the ADMM algorithm.

[0009] The adaptive correlation filter is convolved with the original features of the video frame to obtain the position of the target tracked by the UAV.

[0010] In some embodiments, the step of obtaining the original features of the video frame and performing rotation correction processing to obtain several video frame rotation correction features includes:

[0011] Acquire the video sequence data to be trained and define the minimum rotation angle, maximum rotation angle, and random number;

[0012] The rotation angle range is determined based on the minimum rotation angle and the maximum rotation angle, and the random number is mapped into the rotation angle range to obtain the rotation matrix;

[0013] Obtain the coordinates of feature point elements in the video sequence data to be trained and perform a simulated rotation operation according to the rotation matrix to obtain the coordinates of the new rotation elements.

[0014] The coordinates of the new rotating element are calculated using bilinear interpolation and then cropped to obtain several video frame rotation correction features.

[0015] In some embodiments, constructing adaptive feature residual weights and adaptive regularization coefficients based on a plurality of video frame rotation correction features includes:

[0016] Obtain response information of rotation correction features of adjacent video frames, the response information including response information of rotation correction features of frame t and response information of rotation correction features of frame (t-1);

[0017] The weight matrix of the response information is constructed by a dynamic adaptive residual weighting method. The weight matrix includes the weight matrix of the rotation correction feature of frame t and the weight matrix of the rotation correction feature of frame (t-1).

[0018] The response information of the same frame is multiplied with the weight matrix of the same frame to obtain the calculation result of frame t. The result of frame t-1 is then subtracted to construct the adaptive feature residual weight.

[0019] The degree of response variation is defined based on the degree of change between the response information of the rotation correction feature in frame t and the response information of the rotation correction feature in frame (t-1).

[0020] An adaptive response variation coefficient is constructed based on the degree of response variation, and the adaptive response variation coefficient is embedded into the residual regularization term to construct the adaptive regularization coefficient.

[0021] In some embodiments, constructing the weight matrix of the response information using a dynamic adaptive residual weighting method includes:

[0022] The video frame rotation correction features are offset to obtain offset video frame rotation correction features;

[0023] The average response value of the video frame rotation correction feature and the average response value of the offset video frame rotation correction feature are obtained and the difference is calculated to obtain the local average response residual.

[0024] The weighted summation of the local average response residuals is performed to obtain the global average response residual;

[0025] The local average response residual is combined with the global average response residual to obtain a preliminary weight matrix;

[0026] The initial weight matrix is ​​normalized to obtain the weight matrix of the response information.

[0027] In some embodiments, the expression for the adaptive feature residual weights is as follows:

[0028]

[0029] In the above formula, W represents the adaptive feature residual weights. t The weight matrix represents the rotation correction feature of frame t. W represents the response information of the rotation correction feature in frame t. t-1 The weight matrix represents the rotation correction feature of the (t-1)th frame. This represents the response information of the rotation correction feature in the (t-1)th frame.

[0030] In some embodiments, the adaptive correlation filter is constructed by introducing the adaptive feature residual weights and the adaptive regularization coefficients, and optimizing them using the ADMM algorithm. This includes:

[0031] The adaptive feature residual weights are embedded into the environmental residual term of the correlation filter, and the adaptive regularization coefficients are embedded into the regularization term of the correlation filter to obtain the adaptive correlation filter to be optimized.

[0032] Obtain the objective function of the adaptive correlation filter to be optimized;

[0033] The objective function of the adaptive correlation filter to be optimized is optimized using the ADMM algorithm to construct the adaptive correlation filter.

[0034] In some embodiments, the objective function of the adaptive correlation filter to be optimized is specifically expressed as follows:

[0035]

[0036] In the above formula, Describe the objective function. φ represents the adaptive feature residual weights. t Let f represent the adaptive variation coefficient, and f represent the filter. t-1 Denotes the filter in frame t-1. Let P represent the rotation correction feature, P represent the binary matrix of the cropping feature center region, y represent the predefined class label determined by the Gaussian formula, K represent the total number of feature channels, k represent the k-th channel, and f represent the rotation correction feature. k Let v represent the filter of the k-th channel, and v represent the hyperparameters controlling the learning of the feature residuals.

[0037] In some embodiments, optimizing the objective function of the adaptive correlation filter to be optimized using the ADMM algorithm to construct the adaptive correlation filter includes:

[0038] Auxiliary variables and Lagrange multipliers are introduced and combined with the objective function to construct the augmented Lagrange form of the objective function;

[0039] The augmented Lagrange form of the objective function is subjected to a Fourier transform to obtain the transformed objective function.

[0040] Based on the transformed objective function, the auxiliary variables and the Lagrange multipliers are fixed, and the objective function is solved to obtain the optimal solution of the objective function;

[0041] Based on the transformed objective function, the objective function and the Lagrange multipliers are fixed, and the auxiliary variables are solved to obtain the optimal solution for the auxiliary variables;

[0042] Based on the optimal solutions of the auxiliary variables and the optimal solutions of the objective function, the Lagrange multipliers are updated to obtain the updated Lagrange multipliers.

[0043] A scale step factor is introduced to update the Lagrange multipliers, resulting in the updated Lagrange multipliers.

[0044] A learning rate is introduced to update the rotation-corrected features, resulting in the updated rotation-corrected features;

[0045] The updated Lagrange multipliers, the updated Lagrange multipliers, and the updated rotation correction features are integrated to construct the adaptive correlation filter.

[0046] In some embodiments, the augmented Lagrange form of the objective function is specifically expressed as follows:

[0047]

[0048] In the above formula, Let f represent the augmented Lagrangian form of the objective function, g represent the auxiliary variable, h represent the Lagrange multiplier, δ represent the penalty factor, and φ represent the variable. t Represents the adaptive coefficient of variation. This represents the rotation correction feature, where y represents the predefined class label determined by the Gaussian formula, K represents the total number of feature channels, k represents the k-th channel, and g k Let f represent the auxiliary variable of the k-th channel, λ represent the regularization hyperparameter, and f k Let v represent the filter for the k-th channel, and v represent the hyperparameters controlling the learning of the feature residuals. f represents the adaptive feature residual weights. t-1 Let P be the filter for the (t-1)th frame. T h represents the transposed binary matrix. k Let represent the Lagrange multiplier of the k-th channel.

[0049] To achieve the above objectives, another aspect of this application proposes a UAV target tracking system based on an adaptive correlation filter, the system comprising:

[0050] The preprocessing module is used to obtain the original features of the video frames and perform rotation correction processing to obtain several video frame rotation correction features;

[0051] The construction module is used to construct adaptive feature residual weights and adaptive regularization coefficients based on the rotation correction features of several video frames.

[0052] An optimization module is used to construct an adaptive correlation filter by introducing the adaptive feature residual weights and the adaptive regularization coefficients based on the correlation filter and optimizing them using the ADMM algorithm.

[0053] The target tracking module is used to convolve the adaptive correlation filter with the original features of the video frame to obtain the position of the target tracked by the UAV.

[0054] The embodiments of this application include at least the following beneficial effects: This application provides a UAV target tracking method and system based on an adaptive correlation filter. This scheme obtains several video frame rotation correction features by acquiring the original features of the video frame and performing rotation correction processing. By simulating rotation of the features, the adaptability and generalization ability of the tracker are enhanced, improving the tracking reliability of the tracker under various complex scenarios and targets. Furthermore, based on the correlation filter, adaptive feature residual weights and adaptive regularization coefficients are introduced. The adaptive feature residual weights can enhance the understanding and learning of background features, enabling the filter to better adapt to and cope with different background features, improving the accuracy and stability of the tracker. The adaptive regularization coefficients can achieve the effect of automatically adjusting the learning of historical filters, making more effective use of historical information, improving the accuracy and precision of the tracker, and adapting to complex and ever-changing environments and improving the tracking accuracy of UAVs. Attached Figure Description

[0055] Figure 1 This is a flowchart of a UAV target tracking method based on an adaptive correlation filter provided in an embodiment of this application;

[0056] Figure 2 This is a schematic diagram of the structure of an unmanned aerial vehicle (UAV) target tracking system based on an adaptive correlation filter, provided in an embodiment of this application.

[0057] Figure 3 This is a schematic diagram of the network framework for drone tracking provided in an embodiment of this application;

[0058] Figure 4 This is a schematic diagram of the structure and process of drone tracking provided in an embodiment of this application;

[0059] Figure 5 This is a schematic diagram illustrating the acquisition of local average response residuals provided in an embodiment of this application;

[0060] Figure 6 This is a schematic diagram showing a successful comparison between the tracker of the present invention and an existing tracker provided in this application embodiment;

[0061] Figure 7 This is a schematic diagram comparing the accuracy of the tracker of the present invention with that of an existing tracker, as provided in an embodiment of this application. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of systems and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0063] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0064] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0066] Some shortcomings exist in related technologies, such as the difficulty in identifying targets under rotational changes, and the fact that filters do not fully consider the consistency of background information. Furthermore, fixed learning rates perform poorly in dealing with the dynamic effects of historical information.

[0067] In view of this, this application provides a UAV target tracking method based on an adaptive correlation filter. By introducing a feature rotation correction processing method, it captures target feature information more comprehensively and improves the accuracy of target recognition. Secondly, by combining the weights of local average response residuals and global average response residuals, the differences between features are evaluated, the tracker performance is optimized, and its ability to understand and learn different background features is enhanced. Finally, an adaptive coefficient of variation is introduced to better utilize historical information, improve the accuracy and precision of the tracker, and achieve adaptive adjustment.

[0068] like Figure 3 and Figure 4 As shown, the LRCF proposed in this embodiment uses BACF as a benchmark and improves performance in three aspects. First, a feature processing mechanism is proposed to simulate the potential changes in features by constructing a rotation matrix and linear interpolation. Furthermore, in subsequent model learning, the corrected features replace the original features. Second, it advocates using adaptive feature residual weights to determine the weights of background residuals to mitigate the impact of various environmental changes on filtering performance. In addition, it proposes using adaptive response variability coefficients to dynamically adjust the effects of different historical filters, thereby enhancing the overall robustness of the model.

[0069] Reference Figure 1 , Figure 1 A flowchart of a UAV target tracking method based on an adaptive correlation filter is provided in this embodiment of the invention, referring to... Figure 1 The method includes the following steps:

[0070] S100: Obtain the original features of the video frames and perform rotation correction processing to obtain several video frame rotation correction features;

[0071] It should be noted that common correlation filtering algorithms typically employ feature extraction methods combined with HOG (Homologous Feature Extraction) and directly input them into the DCF (Distributed Dynamics Function) model framework for processing. However, this approach limits the model's generalization ability to some extent, potentially leading to varying adaptability across different scenarios and targets, thus impacting model performance. This invention addresses this by performing rotation correction on the extracted features. By simulating feature rotation, this invention enhances the tracker's adaptability and generalization capabilities, improving tracking reliability across various complex scenarios and targets.

[0072] In some embodiments, step S100 may include steps S110 to S140.

[0073] S110. Obtain the video sequence data to be trained and define the minimum rotation angle, maximum rotation angle and random number;

[0074] In step S110 of some embodiments, the minimum rotation angle is defined as α. min = -V, maximum rotation angle is α min =V. Generate a random number r such that r follows a uniform distribution, i.e., r ~ Uniform(0,1). Use a linear transformation to map the random number r to a random rotation angle α = r × (α) within the desired range. max -α min )+α min .

[0075] S120. Determine the rotation angle range based on the minimum and maximum rotation angles, and map random numbers into the rotation angle range to obtain the rotation matrix;

[0076] In step S120 of some embodiments, a rotation moment R is defined, where α represents the rotation angle:

[0077]

[0078] S130. Obtain the coordinates of feature point elements in the video sequence data to be trained and perform simulated rotation operation according to the rotation matrix to obtain the coordinates of new rotation elements; S140. Calculate the values ​​of the coordinates of the new rotation elements using bilinear interpolation and perform cropping to obtain rotation correction features for several video frames.

[0079] In steps S130 to S140 of some embodiments, for each feature point For each element coordinate (u, o) in the array, perform the following simulated rotation operation:

[0080]

[0081] Each feature point There are D×N elements, therefore we will get D×N rotation coordinates (u ′ ,o ′ ) j j = 1, 2, ..., D × N. Calculate each new rotation element using bilinear interpolation. The value of is expressed as:

[0082]

[0083] where ι=u ′ -u ip ,ρ=O ′ -O ip ,u ′ and O ′ Represents the rotation coordinates, u ip and o ip They represent u respectively ′ and O ′ The integer part. (Symbol) This represents a lower bound function that rounds a real number down to the nearest integer. This represents an upper bound function that rounds a real number up to the nearest integer. Representing coordinates Features The actual value. For The resulting matrix is ​​trimmed to make it compatible with... The sizes are matched to obtain new preprocessed features.

[0084] This approach enables the model to better adapt to target tracking tasks under different poses and scenarios, thereby improving the model's generalization ability. Furthermore, rotation correction feature learning reduces the model's sensitivity to target rotation and pose changes, enhancing its robustness to target pose variations. These improvements give our proposed method superior performance and applicability in the target tracking field. For convenience, we use... This indicates the rotation correction feature in subsequent methods.

[0085] S200. Based on the rotation correction features of several video frames, construct adaptive feature residual weights and adaptive regularization coefficients;

[0086] It should be noted that in some embodiments, step S200 may include steps S210 to S250.

[0087] S210. Obtain the response information of the rotation correction features of adjacent video frames. The response information includes the response information of the rotation correction features of frame t and the response information of the rotation correction features of frame (t-1). S220. Construct a weight matrix of the response information using a dynamic adaptive residual weighting method. The weight matrix includes the weight matrix of the rotation correction features of frame t and the weight matrix of the rotation correction features of frame (t-1).

[0088] It should be noted that in some embodiments, step S220 may include steps S221 to S225.

[0089] S221. Offset the video frame rotation correction feature to obtain the offset video frame rotation correction feature; S222. Obtain the average response value of the video frame rotation correction feature and the average response value of the offset video frame rotation correction feature, and calculate the difference to obtain the local average response residual.

[0090] In steps S221 to S222 of some embodiments, the local average response residual can reflect changes in the target position. This allows the tracker to adjust and correct the target position in a timely manner. This ensures that the tracker can continuously and stably track the target. The local average response residual is defined as... The specific calculation process is as follows: Figure 5 As shown. Where, E t The element in row d and column n Set as:

[0091]

[0092] Δψ represents the offset required to align the peak of the continuous response plot. M t-1 (Δψ) is the offset M t-1 M t-1 The position was changed under the action of the offset Δψ, so that its peak value was similar to M. tConsistent. M t The average response value of frame t is expressed as follows:

[0093]

[0094] in This represents the discrete inverse Fourier transform operator. and They are and Fourier stated that...

[0095] S223. Perform weighted summation on the local average response residuals to obtain the global average response residuals;

[0096] In step S223 of some embodiments, the global average response residual takes into account the differences in the overall target response, and can more completely reflect the changes in the target between adjacent frames, thereby providing more accurate tracking guidance. Global average response residual The expression is as follows:

[0097]

[0098] S224. Combine the local average response residual with the global average response residual to obtain a preliminary weight matrix; S225. Normalize the preliminary weight matrix to obtain the weight matrix of the response information.

[0099] In steps S224 to S225 of some embodiments, the core idea of ​​dynamic adaptive residual weighting is to weight the local average response residual E. t Compared with the global average response residual This combination effectively reduces the impact of sudden environmental changes. When faced with significant environmental changes, the weights must be attenuated to minimize the filter's response to abrupt environmental shifts. Therefore, we first use E... t and calculate in, It means yes The element in row d and column n. The final weight matrix is ​​obtained after normalization. and The specific expression is as follows:

[0100]

[0101] Here, η is used to control the global average residual, with a value of 0.05. This method intelligently adjusts the weights of the feature residuals based on local and global changes in the response. This fine-tuning allows the tracker to operate across different tasks and environments, resulting in significant adaptability and flexibility.

[0102] S230. Multiply the response information of the same frame with the weight matrix of the same frame to obtain the calculation result of frame t and the calculation result of frame t-1, and then subtract them to construct the adaptive feature residual weight.

[0103] In step S230 of some embodiments, environmental information is widely used in correlation filtering algorithms in the field of target tracking to maintain the robustness of tracking. A common method of utilizing environmental information is to introduce an environmental residual regularization term during the filter training phase. However, sudden environmental changes can affect the accuracy and stability of the tracker by influencing the environmental residuals, and the impact of environmental residuals varies under different conditions.

[0104] To mitigate the aforementioned issues, we propose adding weights to the environmental residual term to address the impact of different environments on the tracker. To utilize more response information, we introduce adaptive feature residual weights based on two consecutive frames. Its specific expression is as follows.

[0105]

[0106] Where · is the Hadard product, W t This is the weight matrix. We construct W using a dynamic adaptive residual weighting method. t .

[0107] S240. Define the degree of response variation based on the degree of change between the response information of the rotation correction feature in frame t and the response information of the rotation correction feature in frame (t-1).

[0108] In step S240 of some embodiments, the response variability reveals the changes in the target between two frames, and it increases as the target changes. We define the response variability as... Its element in row d and column n Specifically, it is expressed as follows:

[0109]

[0110] S250. Construct an adaptive response variation coefficient based on the degree of response variation, and embed the adaptive response variation coefficient into the residual regularization term to construct the adaptive regularization coefficient.

[0111] In step S250 of some embodiments, considering that mutations may cause drastic changes in the response, resulting in the loss of informative historical information, we establish a threshold φ0 with a value of 0.3. When the adaptive response variability coefficient exceeds this threshold, φ t Setting it to 0 indicates that the history filter is not included in the learning process. On the other hand, considering that a small change in response often implies the presence of the target, we need to enhance the learning of the history filter. Therefore, we define the expression as follows:

[0112]

[0113] Where μ1 and μ2 represent hyperparameters, Φ t In response to the degree of variation.

[0114] In the field of UAV target tracking, historical information can provide the target's trajectory, improving the tracker's tracking accuracy and adaptability. Taking the Spatiotemporal Regularized Correlation Filter (STRCF) as an example, it can be specifically expressed as follows:

[0115]

[0116] STRCF introduces temporal regularization, successfully tracking targets using historical information even in the presence of occlusion. However, it's noted that the impact of historical information from different frames on filter training varies, especially when responses change drastically, potentially causing abrupt shifts that render historical information unusable. To address this issue, an adaptive response variability coefficient φ is introduced. t This is used to adjust the influence of the history filter on model training.

[0117] S300: Based on the correlation filter, adaptive feature residual weights and adaptive regularization coefficients are introduced, and optimized by the ADMM algorithm to construct an adaptive correlation filter;

[0118] It should be noted that in some embodiments, step S300 may include steps S310 to S330.

[0119] S310. Embed the adaptive feature residual weights into the environmental residual term of the correlation filter, and embed the adaptive regularization coefficients into the regularization term of the correlation filter to obtain the adaptive correlation filter to be optimized.

[0120] S320. Obtain the objective function of the adaptive correlation filter to be optimized;

[0121] In steps S310 to S320 of some embodiments, the present invention enhances the filter's adaptability to abrupt changes by weighting the environmental residual term and adjusting the adaptive parameters. During the problem-solving process, the weights of the environmental residual term and the parameters of the filter regularization term continuously evolve to adapt to changes. The LRCF objective function proposed in this invention is expressed as:

[0122]

[0123] in, φ represents the adaptive feature residual weights, while φ t This represents the adaptive coefficient of variation. This represents the filter for the (t-1)th frame.

[0124] Next, we introduced (auxiliary variables). and Where s is the Lagrange multiplier and δ is the penalty factor. The augmented Lagrange form of the objective function is expressed as:

[0125]

[0126] Using Parseval's theorem, the augmented Lagrange form of the objective function is rewritten in the Fourier domain and then transformed into matrix form, resulting in the following expression:

[0127]

[0128] in, ∧ It is the corresponding representation after the discrete Fourier transform.

[0129] S330. The objective function of the adaptive correlation filter to be optimized is optimized using the ADMM algorithm to construct the adaptive correlation filter.

[0130] It should be noted that in some embodiments, step S330 may include steps S331 to S338.

[0131] S331. Introduce auxiliary variables and Lagrange multipliers, and merge them with the objective function to construct the augmented Lagrange form of the objective function; S332. Perform a Fourier transform on the augmented Lagrange form of the objective function to obtain the transformed objective function; S333. Based on the transformed objective function, fix the auxiliary variables and Lagrange multipliers, and solve the objective function to obtain the optimal solution; S334. Based on the transformed objective function, fix the objective function and Lagrange multipliers, and solve the auxiliary variables to obtain the optimal solution. S335. Based on the optimal solutions of the auxiliary variables and the objective function, update the Lagrange multipliers to obtain the updated Lagrange multipliers; S336. Introduce a scale step factor to update the Lagrange multipliers to obtain the updated Lagrange multipliers; S337. Introduce a learning rate to update the rotation-corrected features to obtain the updated rotation-corrected features; S338. Integrate the updated Lagrange multipliers, updated Lagrange multipliers, and updated rotation-corrected features to construct an adaptive correlation filter.

[0132] In some specific embodiments, the augmented Lagrangian form of the objective function is converted into a matrix form and decomposed into multiple subproblems for solving using the ADMM algorithm.

[0133] Subproblem f: fixed and get The optimal solution is as follows:

[0134]

[0135] In the above formula, f represents the discrete Fourier transform. Let g represent the discrete Fourier transform. Let h represent the result of the discrete Fourier transform.

[0136] Solving formula The closed-form solution to f is obtained, and its expression is:

[0137]

[0138] Subproblem g: fixed and get The optimal solution is as follows:

[0139]

[0140] The approximate solution obtained using the Sherman-Morrison formula is:

[0141]

[0142] Subproblem h: Lagrange multiplier update as follows:

[0143]

[0144] Subproblem δ: The Lagrange multiplier is updated as follows:

[0145] δ (i+1) =min(βδ) i ,δ max )

[0146] Where, δ max δ is the maximum value, and β is the scale step factor.

[0147] Feature update

[0148]

[0149] Where θ represents the learning rate.

[0150] S400: Convolve the adaptive correlation filter with the original features of the video frame to obtain the position of the target tracked by the UAV;

[0151] In summary, this invention incorporates rotation-corrected features into the model training. During the training phase, a weight matrix and regularization coefficients are constructed based on the response information from two consecutive frames. This improvement enables the filter to adaptively adjust during real-time tracking. Furthermore, an efficient Alternating Direction Multiplier (ADMM) is used to optimize the tracker. Finally, the trained filter is convolved with the features of the new frame to obtain the target's location. Therefore, this invention offers the following advantages:

[0152] 1) The extracted features were subjected to rotation correction. By simulating the rotation of the features, the tracker's adaptability and generalization ability are enhanced, thereby improving the tracker's tracking reliability in various complex scenarios and targets.

[0153] 2) By constructing weights for feature residuals using a combination of mean squared error of the combined response and global average response residuals, the differences between features are evaluated, enhancing the understanding and learning of background features. This allows the filter to better adapt to and cope with different background features, improving the accuracy and stability of the tracker.

[0154] 3) By comparing the response information of consecutive frames to construct an adaptive coefficient of variation, the learning effect of automatically adjusting the historical filter is achieved. This method utilizes historical information more effectively, improving the accuracy and precision of the tracker.

[0155] Furthermore, the proposed LRCF was compared with several advanced trackers on authoritative UAV benchmarks (UAV123@10FPS and UAVDT). For benchmarking, the single-shot evaluation (OPE) criterion was chosen to measure the LRCF's tracking accuracy and success rate, with frames per second (FPS) selected as the metric for tracking speed. The comparisons demonstrate that the proposed LRCF exhibits superior performance.

[0156] It should be noted that the UAV123@10fps dataset contains 123 high-definition video sequences captured by low-altitude drones. These videos contain numerous challenging tracking scenarios, such as prolonged full and partial occlusion and camera movement. These challenges require the tracking algorithm to possess good robustness and adaptability, enabling it to stably track targets in complex environments and ensuring tracking accuracy and stability.

[0157] The UAVDT dataset consists of video sequences captured by drones at multiple locations in urban areas, covering common scenes such as squares and main roads, while also containing a large number of small objects and complex backgrounds, presenting new challenges for target tracking tasks. These diverse and complex scenes require tracking algorithms to have strong robustness and accuracy, effectively handling background interference and target occlusion.

[0158] Furthermore, we performed a quantitative analysis of the experimental results. On a UAV123 at 10fps, we compared the proposed LRCF with 11 trackers, including ECO, BACF, STRCF, AutoTrack, BiCF, ReCF, EMCF, RCFL, RBSCF, EFSCF, and MSEFCF. This was to confirm the effectiveness of the invention. Figure 6 and Figure 7 As can be seen, the tracking success rate of LRCF is as high as 60.0%, which is 9.4% higher than the benchmark method BACF. The tracking accuracy of LRCF is as high as 69.6%, which is 12.4% higher than the benchmark method BACF. Experimental results show that the LRCF proposed in this invention has achieved significant results in improving tracking performance.

[0159] The proposed LRCF was compared with 11 trackers on challenging properties. Table 1 shows that the LRCF achieved the highest tracking accuracy in aspect ratio variation (ARC), low resolution (LR), out-of-field (OV), illumination variation (IV), and camera motion (CM). The experimental results fully demonstrate that the proposed LRCF has high robustness and can cope with complex tracking environments.

[0160] Table 1 shows the accuracy of LRCF and 9 different attribute trackers on UAV123@10fps.

[0161]

[0162]

[0163] On the UAVDT dataset, we selected 19 deep learning trackers for comparative experiments. These deep learning methods include ADNet, DeepSTRCF, MCCT, TADT, UDT+, MN_ECO, MN_MDNet, fECO, LUDT+, MEVT, SE-SiamFC, HiFT, STARK-ST101, CCF, CCF-L, BSTCF, RTDG, RCFL, and MSEFCF. As shown in Table 2, although LRCF achieves a high accuracy of 73.8% and a speed of 69.61%, the experimental results demonstrate that LRCF exhibits good performance.

[0164] Table 2 compares the performance of LRCF with other excellent depth trackers on the UAVDT benchmark.

[0165]

[0166]

[0167] Finally, an ablation study was conducted. This embodiment of the invention compared four methods on a UAV123 at 10fps. We chose BACF as the baseline. [The text then abruptly shifts to a different topic:] ...in the baseline... ad In the baseline, we add adaptive feature residual weights and adaptive coefficient of variation. or Introducing rotation feature learning. ad+or Adaptive residual weights and adaptive response variance coefficients are added during rotation feature learning.

[0168] Table 3 shows the experimental results, indicating that on this dataset, compared to the other three methods, baseline... ad+or Both tracking accuracy and tracking success rate showed the best level.

[0169] Table 3. Study on ablation results of three methods based on baseline.

[0170] baseline 0.572 0.506 0.478 0.430 0.532 0.467 0.491 0.605 <![CDATA[baseline ad ]]> 0.679 0.596 0.614 0.580 0.658 0.596 0.601 0.700 <![CDATA[baseline or ]]> 0.690 0.587 0.613 0.572 0.646 0.596 0.598 0.686 <![CDATA[baseline ad+or ]]> 0.696 0.600 0.626 0.608 0.671 0.608 0.602 0.701

[0171] Specifically, compared to baseline, baseline ad+or Tracking accuracy improved by 1.7%, and success rate improved by 0.4%. Furthermore, the baseline... ad+orIt outperforms the other three methods in accuracy across six attributes (ARC, IV, CM, PO, VC, and SO). Table 3 provides strong support for the effectiveness of LRCF, further demonstrating the excellent performance of the improved tracker LRCF.

[0172] Please see Figure 2 This application also provides a UAV target tracking system based on an adaptive correlation filter, which can implement the above-mentioned UAV target tracking method based on an adaptive correlation filter. The system includes:

[0173] Preprocessing module 201 is used to acquire the original features of video frames and perform rotation correction processing to obtain several video frame rotation correction features;

[0174] Construction module 202 is used to construct adaptive feature residual weights and adaptive regularization coefficients based on the rotation correction features of several video frames;

[0175] Optimization module 203 is used to construct an adaptive correlation filter by introducing the adaptive feature residual weights and the adaptive regularization coefficients based on the correlation filter and optimizing them using the ADMM algorithm.

[0176] The target tracking module 204 is used to convolve the adaptive correlation filter with the original features of the video frame to obtain the position of the target tracked by the UAV.

[0177] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0178] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A UAV target tracking method based on an adaptive correlation filter, characterized in that, The method includes the following steps: The original features of the video frames are obtained and rotation correction is performed to obtain several video frame rotation correction features. Based on several video frame rotation correction features, adaptive feature residual weights and adaptive regularization coefficients are constructed; Based on the correlation filter, the adaptive feature residual weights and the adaptive regularization coefficients are introduced, and the adaptive correlation filter is constructed by optimizing it through the ADMM algorithm. The adaptive correlation filter is convolved with the original features of the video frame to obtain the position of the target tracked by the UAV. The step of constructing adaptive feature residual weights and adaptive regularization coefficients based on several video frame rotation correction features includes: Obtain response information of rotation correction features of adjacent video frames, the response information including response information of rotation correction features of frame t and response information of rotation correction features of frame (t-1); The weight matrix of the response information is constructed by a dynamic adaptive residual weighting method. The weight matrix includes the weight matrix of the rotation correction feature of frame t and the weight matrix of the rotation correction feature of frame (t-1). The response information of the same frame is multiplied with the weight matrix of the same frame to obtain the calculation result of frame t. The result of frame t-1 is then subtracted to construct the adaptive feature residual weight. The degree of response variation is defined based on the degree of change between the response information of the rotation correction feature in frame t and the response information of the rotation correction feature in frame (t-1). An adaptive response variation coefficient is constructed based on the degree of response variation, and the adaptive response variation coefficient is embedded into the residual regularization term to construct the adaptive regularization coefficient.

2. The method according to claim 1, characterized in that, The process of acquiring the original features of video frames and performing rotation correction processing yields several video frame rotation correction features, including: Obtain the original features of the video frame and define the minimum rotation angle, maximum rotation angle, and random number; The rotation angle range is determined based on the minimum rotation angle and the maximum rotation angle, and the random number is mapped into the rotation angle range to obtain the rotation matrix; Obtain the coordinates of feature point elements of the original features of the video frame and perform a simulated rotation operation according to the rotation matrix to obtain the coordinates of the new rotation elements. The coordinates of the new rotating element are calculated using bilinear interpolation and then cropped to obtain several video frame rotation correction features.

3. The method according to claim 2, characterized in that, The construction of the weight matrix for the response information using the dynamic adaptive residual weighting method includes: The video frame rotation correction features are offset to obtain offset video frame rotation correction features; The average response value of the video frame rotation correction feature and the average response value of the offset video frame rotation correction feature are obtained and the difference is calculated to obtain the local average response residual. The weighted summation of the local average response residuals is performed to obtain the global average response residual; The local average response residual is combined with the global average response residual to obtain a preliminary weight matrix; The initial weight matrix is ​​normalized to obtain the weight matrix of the response information.

4. The method according to claim 1, characterized in that, The specific expression for the adaptive feature residual weights is as follows: In the above formula, Represents the adaptive feature residual weights. The weight matrix represents the rotation correction feature of frame t. This represents the response information of the rotation correction feature in frame t. The weight matrix represents the rotation correction feature of the (t-1)th frame. This represents the response information of the rotation correction feature in the (t-1)th frame.

5. The method according to claim 1, characterized in that, The adaptive correlation filter, based on the correlation filter, incorporates the adaptive feature residual weights and the adaptive regularization coefficients, and is optimized using the ADMM algorithm to construct an adaptive correlation filter, including: The adaptive feature residual weights are embedded into the environmental residual term of the correlation filter, and the adaptive regularization coefficients are embedded into the regularization term of the correlation filter to obtain the adaptive correlation filter to be optimized. Obtain the objective function of the adaptive correlation filter to be optimized; The objective function of the adaptive correlation filter to be optimized is optimized using the ADMM algorithm to construct the adaptive correlation filter.

6. The method according to claim 5, characterized in that, The specific expression of the objective function of the adaptive correlation filter to be optimized is as follows: In the above formula, Describe the objective function. Represents the adaptive feature residual weights. Represents the adaptive coefficient of variation. Indicates a filter. Indicates the first Frame filter, Indicates rotational correction features. A binary matrix representing the central region of the clipping feature. This represents a predefined class label determined by Gauss's formula. Indicates the total number of feature channels. Indicates the first One channel, Indicates the first Filters for each channel This represents the hyperparameters that control the learning of feature residuals.

7. The method according to claim 5, characterized in that, The step of optimizing the objective function of the adaptive correlation filter to be optimized using the ADMM algorithm to construct the adaptive correlation filter includes: Auxiliary variables and Lagrange multipliers are introduced and combined with the objective function to construct the augmented Lagrange form of the objective function; The augmented Lagrange form of the objective function is subjected to a Fourier transform to obtain the transformed objective function. Based on the transformed objective function, the auxiliary variables and the Lagrange multipliers are fixed, and the objective function is solved to obtain the optimal solution of the objective function; Based on the transformed objective function, the objective function and the Lagrange multipliers are fixed, and the auxiliary variables are solved to obtain the optimal solution for the auxiliary variables; Based on the optimal solutions of the auxiliary variables and the optimal solutions of the objective function, the Lagrange multipliers are updated to obtain the updated Lagrange multipliers. A scale step factor is introduced to update the Lagrange multipliers, resulting in the updated Lagrange multipliers. A learning rate is introduced to update the rotation-corrected features, resulting in the updated rotation-corrected features; The updated Lagrange multipliers, the updated Lagrange multipliers, and the updated rotation correction features are integrated to construct the adaptive correlation filter.

8. The method according to claim 7, characterized in that, The augmented Lagrange form of the objective function is expressed as follows: In the above formula, This represents the augmented Lagrange form of the objective function. Indicates a filter. Represents auxiliary variables. Represents the Lagrange multipliers. Indicates the penalty factor. Represents the adaptive coefficient of variation. Indicates rotational correction features. This represents a predefined class label determined by Gauss's formula. Indicates the total number of feature channels. Indicates the first One channel, Indicates the first Auxiliary variables for each channel, This represents the regularization hyperparameter. Indicates the first Filters for each channel This represents the hyperparameters that control the learning of feature residuals. Represents the adaptive feature residual weights. Indicates the first Frame filters This represents the transposed binary matrix. Indicates the first Lagrange multipliers for each channel.

9. A UAV target tracking system based on an adaptive correlation filter, characterized in that, The system includes: The preprocessing module is used to obtain the original features of the video frames and perform rotation correction processing to obtain several video frame rotation correction features; The construction module is used to construct adaptive feature residual weights and adaptive regularization coefficients based on the rotation correction features of several video frames. An optimization module is used to construct an adaptive correlation filter by introducing the adaptive feature residual weights and the adaptive regularization coefficients based on the correlation filter and optimizing them using the ADMM algorithm. The target tracking module is used to convolve the adaptive correlation filter with the original features of the video frame to obtain the position of the target tracked by the UAV. The step of constructing adaptive feature residual weights and adaptive regularization coefficients based on several video frame rotation correction features includes: Obtain response information of rotation correction features of adjacent video frames, the response information including response information of rotation correction features of frame t and response information of rotation correction features of frame (t-1); The weight matrix of the response information is constructed by a dynamic adaptive residual weighting method. The weight matrix includes the weight matrix of the rotation correction feature of frame t and the weight matrix of the rotation correction feature of frame (t-1). The response information of the same frame is multiplied with the weight matrix of the same frame to obtain the calculation result of frame t. The result of frame t-1 is then subtracted to construct the adaptive feature residual weight. The degree of response variation is defined based on the degree of change between the response information of the rotation correction feature in frame t and the response information of the rotation correction feature in frame (t-1). An adaptive response variation coefficient is constructed based on the degree of response variation, and the adaptive response variation coefficient is embedded into the residual regularization term to construct the adaptive regularization coefficient.

Images