Unmanned aerial vehicle aerial photography long-term target tracking method based on matching loss confidence

Abstract

The invention relates to the technical field of unmanned aerial vehicle aerial photography and computer vision, in particular to an unmanned aerial vehicle aerial photography long-term target tracking method based on matching loss confidence. Comprising the steps of performing multi-feature fusion extraction on a tracking target object, calculating the similarity between a target template and a candidate region, performing target search matching by using an improved pelican search algorithm, realizing target scale adaptive updating through a scale pyramid, and dynamically updating the target template based on an adaptive learning rate; and designing a long-term tracker combined with the matching loss confidence, wherein the long-term tracker comprises the steps of judging a target tracking state based on the matching loss confidence and executing rapid positioning redetection when the target is lost. Through multi-feature fusion, an improved pelican search algorithm and a matching loss confidence judgment mechanism, the long-term stable tracking of the target in the aerial photographing scene of the unmanned aerial vehicle is realized, the problems of shielding, scale change, target loss and the like are effectively solved, and the tracking precision, the success rate and the real-time performance are considered.

Description

Technical Field

[0001] This invention relates to the fields of drone aerial photography and computer vision technology, and in particular to a long-term target tracking method for drone aerial photography based on matching loss confidence. Background Technology

[0002] Compared to short-term target tracking, long-term target tracking in UAV aerial photography faces more severe challenges, including prolonged target occlusion, target re-identification, and the accuracy of predicting long-term motion trends. In occluded scenarios, appearance and motion features may become unreliable, increasing the risk of association failure. For targets that have been lost for a long time, predicting their long-term motion is severely affected by accumulated errors, making target re-identification challenging. Furthermore, long-term tracking needs to handle interference factors such as target scale changes, rapid movement, and lighting variations. These problems may not be obvious in short-term tracking but become critical issues that urgently need to be addressed in long-term tracking. The swarm intelligence algorithm SI can flexibly handle different types of optimization problems and performs well in adapting to environmental changes in complex scenarios. Leveraging its powerful global search capabilities and precise local localization capabilities, SI can locate targets within the search space, providing a solution to the bottlenecks of long-term tracking algorithms. Summary of the Invention

[0003] To address the problems of occlusion, difficulty in recovering lost targets, and poor adaptation to scale changes in long-term target tracking during UAV aerial photography, this invention proposes a long-term target tracking method for UAV aerial photography based on matching loss confidence. Combining the optimization capabilities of the improved Pelican Search algorithm with the representational advantages of multi-feature fusion, a high-precision and robust target tracking framework is designed to improve the accuracy and stability of long-term tracking in complex scenarios.

[0004] To achieve the above objectives, the technical solution adopted is:

[0005] This invention proposes a long-term target tracking method for UAV aerial photography based on matching loss confidence, comprising the following steps:

[0006] The design of the target tracking framework includes multi-feature fusion extraction of the target object, calculation of the similarity between the target template and the candidate region, target search and matching using the improved pelican search algorithm, adaptive updating of the target scale through the scale pyramid, and dynamic updating of the target template based on the adaptive learning rate.

[0007] Design a long-term tracker with joint matching loss confidence, including determining the target tracking status based on the matching loss confidence and performing fast localization and re-detection when the target is lost.

[0008] According to the UAV aerial long-term target tracking method based on matching loss confidence of the present invention, the multi-feature fusion extraction of the tracked target object further includes: extracting CN features, FHOG features, and Pool4 layer features of the target region, and performing weighted fusion according to the following formula to obtain the fused multidimensional feature γ. t :

[0009] γ t =ω CN γ CN +ω FHOG γ FHOG +ω Pool4 γ Pool4

[0010] Where, γ CN γ FHOG and γ Pool4 These represent the CN features, FHOG features, and Pool4 layer features of the target object, respectively, ω CN ω FHOG and ω Pool4 These are the feature fusion coefficients corresponding to different features; the fused multidimensional feature γ t Used to construct the feature model of the target template and the feature representation of the candidate region in the search phase.

[0011] According to the UAV aerial long-term target tracking method based on matching loss confidence of the present invention, the calculation of the similarity between the target template and the candidate region further includes: calculating the similarity between the target template feature model p and the candidate region feature model q based on the Bhattacharyya coefficient, and the calculation formula is as follows:

[0012]

[0013] Where p = {p1, p2, ..., p} θ}, q={q1, q2, ..., q θ}, θ1 is the quantization level, p i and q i These represent the probabilities of the i-th feature appearing in the two feature models, respectively; the value of the Bhattacharyya coefficient B(p,q) serves as the fitness evaluation function of the improved pelican search algorithm in the target search matching process.

[0014] According to the UAV aerial long-term target tracking method based on matching loss confidence of the present invention, the target search and matching using the improved pelican search algorithm further includes:

[0015] An improvement to the short-term memory module based on Gaussian distribution is made by replacing the initialization of the algorithm's initial population with a Gaussian distribution based on the prediction of the target's motion trajectory, instead of a uniform distribution.

[0016] Based on the improved reproduction correction strategy of survival of the fittest, the reproduction range and number of offspring are dynamically determined according to the fitness of parent individuals, and the population is updated through secondary selection.

[0017] Based on the improved fusion differential directional mutation strategy, a decision coefficient ζ and a determination coefficient c are introduced to dynamically select whether to perform mutation updates based on the current individual position or the global optimal position, so as to balance the algorithm's global exploration and local development capabilities.

[0018] According to the UAV aerial long-term target tracking method based on matching loss confidence of the present invention, the improvement of the short-term memory module based on Gaussian distribution further includes:

[0019] Based on the optimal target position X obtained from the first two frames best(t-1) and X best(t-2) Calculate the mean μ of the population distribution in frame t. t =X best(t-1) +(X best(t-1) -X best(t-2) ), and based on this mean μ t and the covariance matrix χ of the population initialized in frame t t Construct a Gaussian distribution model to generate the initial population individual positions in frame t. This causes the initial population to concentrate in the area where the target may appear.

[0020] According to the UAV aerial long-term target tracking method based on matching loss confidence of the present invention, the improved position update rule based on the fusion differential directional mutation strategy is further as follows:

[0021]

[0022] Where τ is the perturbation factor, generated by the gamma random jump coefficient, and X best It is the current global optimal individual position. Let X be the updated position vector. i,j Let t be the current position vector of the individual, t be the current iteration number, T be the maximum iteration number, rand be a uniformly distributed random number, and R be the exploration coefficient.

[0023] According to the UAV aerial long-term target tracking method based on matching loss confidence of the present invention, when the target scale is adaptively updated through a scale pyramid, the candidate region size selection formula is as follows: Among them, M i and N iLet be the length and width of the target in the i-th frame image, respectively, and η be the scale factor, n s s is the scale factor index, and s is the number of scale factors. Multi-scale candidate regions with the same aspect ratio are constructed to achieve scale prediction.

[0024] Additionally, a two-dimensional scale perception factor c is introduced. i =(c xi ,c yi The formula for selecting the candidate region size is updated to: (This is used to adjust the aspect ratio of the target bounding box.) Where c xi and c yi n represents the scale factor of the bounding box along the x-axis and y-axis, respectively. i ,n j An independent index for aspect ratio changes.

[0025] According to the UAV aerial long-term target tracking method based on matching loss confidence of the present invention, the step of dynamically updating the target template based on adaptive learning rate further includes:

[0026] The formula for calculating the adaptive learning rate is as follows: Where φ(t) is the learning rate of the feature template at frame t, φ max It is the peak learning rate, f i γ(t) represents the fitness of the i-th individual, n is the population size, and δ1 and δ2 are adjustment parameters; the target template update strategy is: γ(t) = [1-φ(t)]γ(t-1) + φ(t)γ best (t), where γ(t) is the target feature template of the t-th frame, γ best (t) is the optimal target feature template for frame t.

[0027] According to the UAV aerial long-term target tracking method based on matching loss confidence of the present invention, the step of determining the target tracking status based on matching loss confidence includes:

[0028] Define the comprehensive loss function L totle =τ cls L cls +τ reg L reg L cls For classification loss, L reg For regression loss, τ cls and τ reg For adjustment coefficients;

[0029] Based on the aforementioned comprehensive loss function L totle The tracking confidence C of frame t is obtained by modeling using a Gaussian function. t =exp(-γ t Ltotle 2 ), where γ t Parameters used to control the rate at which confidence levels decrease;

[0030] Set a confidence threshold θ and define a tracker switching factor Q. t (5) If the tracking confidence C of 5 consecutive frames t If all values ​​are below the threshold θ, then Q t (5) =1, indicating the target is lost, and the search tracker is initiated to perform a re-detection process; otherwise, Q t (5) =0, indicating that the tracking status is reliable, and the basic tracker continues to execute the tracking task.

[0031] According to the UAV aerial long-term target tracking method based on matching loss confidence of the present invention, the step of performing fast localization and re-detection when the target is lost includes:

[0032] Based on a Gaussian distribution-based short-term memory module, a search population for the improved pelican search algorithm is initialized within the estimated possible target region. Individual population members contain the target's location and scale information.

[0033] For each candidate region in the search population, its multi-feature fusion representation is extracted, and the similarity with the target template is calculated as the fitness.

[0034] Using the improved pelican search algorithm, the population is iteratively updated based on the fitness, and the location and scale parameters of the target are optimized until the optimal candidate target is obtained, thus completing the relocation and retrieval of the lost target.

[0035] The beneficial effects achieved by adopting the above technical solution are:

[0036] To address the long-term tracking problem in UAV aerial target tracking, this invention proposes a target tracking framework based on an improved Pelican Search algorithm. An adaptive weighting factor is designed to fuse handcrafted CN features, FHOG features, and Pool4-layer features to obtain enhanced multi-dimensional fusion features. The Bhattacharyya coefficient is selected as the similarity function to evaluate the matching degree between the target template and candidate regions. Simultaneously, a template learning mechanism is designed to dynamically adapt to changes in target appearance, continuously learning and updating the latest target features. The Pelican Search algorithm is optimized and improved to enhance its global optimization and local localization capabilities. Experiments on standard test functions show that the improved Pelican Search algorithm significantly improves optimization ability and convergence speed compared to the original version. Furthermore, the target tracking framework proposed in this invention exhibits strong robustness and adaptability in video sequences, significantly improving the accuracy and stability of target tracking, and effectively addressing challenges such as prolonged target occlusion and re-identification after loss during tracking. Attached Figure Description

[0037] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings of the embodiments of the present invention will be briefly described below. The drawings are merely illustrative of some embodiments of the present invention and are not intended to limit the scope of the present invention to all embodiments.

[0038] Figure 1 This is the framework of the UAV aerial photography long-term target tracking method based on matching loss confidence in this embodiment of the invention;

[0039] Figure 2 This is an initialization effect diagram of the short-term memory module according to an embodiment of the present invention;

[0040] Figure 3 This is a schematic diagram illustrating the generation of pelican offspring by the propagation and transmission operator in an embodiment of the present invention;

[0041] Figure 4 This is a target tracking framework based on swarm intelligence algorithms, as described in this invention.

[0042] Figure 5 This is a flowchart of the target tracking framework based on the improved pelican search algorithm according to an embodiment of the present invention;

[0043] Figure 6 This is a diagram illustrating the dimensional changes under a fixed aspect ratio according to an embodiment of the present invention.

[0044] Figure 7 This is a diagram illustrating the scale ratio perception factor effect of an embodiment of the present invention.

[0045] Figure 8 This is a flowchart of the long-term stable tracking method for drone aerial targets according to an embodiment of the present invention;

[0046] Figure 9 These are the accuracy (a) and success rate (b) graphs of the six algorithms in this embodiment of the invention on the UAV123 dataset. Detailed Implementation

[0047] The exemplary solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art.

[0048] like Figure 1 As shown, this invention discloses a long-term target tracking method for UAV aerial photography based on matching loss confidence. A target tracking state judgment module is designed based on matching loss confidence to determine the target state during tracking. When the tracking state is reliable, a target tracking algorithm based on a Transformer network is used as the basic tracker to perform stable and fast target tracking. When the tracking state is unreliable, the method switches to a target tracker based on an improved pelican search algorithm to search for the target within the search space, and integrates scale perception factors and fast localization to retrieve the target. This method specifically includes two steps:

[0049] Step S1: Design the target tracking framework, including multi-feature fusion extraction of the target object, calculation of the similarity between the target template and the candidate region, target search and matching using the improved pelican search algorithm, adaptive updating of the target scale through a scale pyramid, and dynamic updating of the target template based on an adaptive learning rate. Each sub-step is described in detail below.

[0050] Step S101: Perform multi-feature fusion extraction on the tracked target object.

[0051] While considering the CN and FHOG features of the target region, features from the Pool4 layer of the VGG-19 network were further extracted to better extract multidimensional features and handle illumination variations and deformation issues. The resulting fused features are as follows:

[0052] γ t =ω CN γ CN +ω FHOG γ FHOG +ω Pool4 γ Pool4

[0053] Where, γ t For the fused multidimensional features, γ CN γ FHOG and γ Pool4 These represent the CN features, FHOG features, and Pool4 layer features of the target object, respectively, ωCN ω FHOG and ω Pool4 These are the feature fusion coefficients corresponding to different features; the fused multidimensional feature γ t Used to construct the feature model of the target template and the feature representation of the candidate region in the search phase.

[0054] Step S102: Calculate the similarity between the target template and the candidate region.

[0055] The similarity between the target template feature model p and the candidate region feature model q is calculated based on the Bhattacharyya coefficient. The calculation formula is as follows:

[0056]

[0057] Where p is the target template feature model obtained by multi-feature fusion of the template image in step S101, defined as p = {p1, p2, ..., p...} θ}, where q is the candidate region feature model obtained by multi-feature fusion of candidate regions in step S101, defined as q = {q1, q2, ..., q}. θ}, θ1 is the quantization level, p i and q i q and b represent the probabilities of the i-th feature appearing in the two feature models, respectively. The value of B(p,q) is proportional to the similarity and serves as the fitness evaluation function of the improved Pelican Search algorithm in the target search matching process in step S103.

[0058] Step S103: Perform target search and matching using the improved Pelican Search algorithm. This step specifically includes sub-steps S1031 to S1033.

[0059] Step S1031: Improve the short-term memory module based on Gaussian distribution by replacing the initial population of the algorithm with a Gaussian distribution initialization based on the prediction of the target motion trajectory.

[0060] The original uniform distribution initialization is replaced with Gaussian distribution-based initialization. The positional changes of the target object in adjacent frames are analyzed to predict possible motion trajectories, resulting in a more concentrated initial population distribution area within the predicted region. The improved formula for calculating the initial number of pelicans is as follows:

[0061]

[0062] in, This represents the i-th individual in the initial population of the t-th frame image. For individuals The probability that follows a Gaussian distribution, χ t Initialize the covariance matrix of the population for frame t; μ tThe mean of the population distribution in frame t is related to the predicted position, and is calculated using the following formula:

[0063] μ t =X best(t-1) +ν t

[0064] v t =X best(t-1) -X best(t-2)

[0065] Among them, X best(t-1) and X best(t-2) v represents the optimal target position (optimal solution) obtained by the improved Pelican Search algorithm in frames t-1 and t-2, respectively. t The distance between two consecutive frames is the target distance.

[0066] Figure 2 This image shows the initialization effect of a Gaussian distribution-based short-term memory module. This module calculates the target's movement speed in the current frame based on the target's movement distance in the previous two frames, then sets the calculated value as the expected parameter of the Gaussian distribution model to obtain the initial distribution coordinates of the pelican population. Employing a Gaussian distribution-based initialization strategy concentrates the population in the area most likely to contain the target object, improving search efficiency while preserving the target object's movement speed characteristics, thus providing a certain degree of target trajectory prediction capability.

[0067] In the SI algorithm, the initialization method significantly impacts both the solution outcome and the algorithm's efficiency. In target tracking tasks, the probability of a target's position changing drastically within consecutive frames is typically low. Therefore, random initialization may reduce the real-time performance of the tracking task. To address this issue, this invention proposes a short-term memory module based on the Gaussian distribution principle. It replaces the original uniform initialization with Gaussian-based initialization, analyzing the positional changes of the target object between adjacent frames to predict possible motion trajectories, thus concentrating the initial population distribution area more closely within the predicted region. This strategy fully leverages the pelican's global exploration capabilities, locking onto the optimal solution in a local area as quickly as possible and reducing the exploration of ineffective regions.

[0068] Step S1032: Based on the improved reproduction correction strategy of survival of the fittest, the reproduction range and number of offspring population are dynamically determined according to the fitness of parent individuals, and the population is updated through secondary selection.

[0069] First, based on the pelican's foraging and circling flight characteristics, an ellipse is selected to simulate the range of pelican breeding and the emergence of a new population. The breeding operator is then determined, and the axis length L of the resulting pelican offspring population range is calculated. i and S i And the number of Pelicans Q i The calculation formula is as follows:

[0070]

[0071]

[0072] Among them, f best and f worst d1 and d2 represent the best and worst fit individuals in the current pelican population, respectively; d1 and d2 are coefficients used to adjust the axis length; σ is the adaptive mutation factor, defining the intensity of gene mutation during the inheritance process at different iteration periods. A larger σ indicates less similarity between offspring and parents, resulting in a stronger global search capability; a smaller σ indicates greater similarity between offspring and parents, leading to higher search accuracy. The formula for calculating σ is as follows:

[0073] σ i =α*(1-i / T) max )

[0074] Where i is the current iteration number; T max The maximum number of iterations is denoted by α, which is the given initial mutation magnitude. As the number of iterations increases, the mutation magnitude gradually decreases, and the disturbance to the pelican population gradually diminishes. This can effectively improve the global search capability and local search accuracy of the improved pelican search algorithm.

[0075] Then, the best offspring are selected from the offspring pelican population obtained through breeding, and the best offspring are compared with the parent pelicans. A new pelican population is obtained through secondary selection, which increases the diversity of the algorithm.

[0076] Figure 3 This diagram illustrates the generation of pelican offspring using the propagation operator. The optimal offspring are selected from the resulting pelican population, and their quality is compared with that of their parent pelicans. A second selection process yields a new pelican population. An improved pelican search algorithm navigates among the different offspring populations to find the optimal pelican, refining the current population for further optimization. In the diagram, F represents the original pelican population. mi A pelican population is a descendant population that reproduces from a single parent pelican. Its size and number of offspring depend on the genetic superiority of the parent pelican. Parents with better environmental adaptability and superior genes produce a wider range and greater number of offspring. new The new pelican population obtained after the second selection process has a spatially uniform distribution of offspring locations, and the memory transfer rule is defined as follows:

[0077]

[0078] Where A is the allowable threshold for the vector difference between the offspring pelican population and individual parent pelicans, used to limit the boundary of the offspring's positional offset relative to the parent during the breeding correction process, ensuring the stability and effectiveness of population renewal; F M and F m These are the vector representations of the parent and offspring pelican individuals, respectively. The essence of the breeding correction is to generate uniformly distributed random offsets in each dimension based on the fitness of individuals in the original pelican population, thereby obtaining a completely new pelican population and increasing the diversity of the algorithm.

[0079] Step S1033: Based on the improved fusion differential directional mutation strategy, a decision coefficient ζ and a determination coefficient c are introduced. When dynamically selecting, mutation updates are performed based on the current individual position or the global optimal position, so as to balance the global exploration and local development capabilities of the algorithm.

[0080] In the target search and matching process, fusion features are extracted for each candidate region corresponding to a candidate position within the search area according to step S101, and fitness values ​​are calculated based on the similarity function in step S102. These fitness values ​​guide the improved pelican search algorithm to iteratively update candidate positions. Based on this, the present invention, in addition to directed mutation, designs a determination coefficient ζ and a decision coefficient c, which are used to determine the relationship between the current pelican position X(i,j) and the current generation's globally optimal pelican position X. best The choice is made between these two values. If c is greater than ζ, the position is updated based on the current pelican position X(i,j); if c is less than or equal to ζ, the position is updated based on the current globally optimal pelican position X. best Move. The improved Pelican Search algorithm's position update expression is as follows:

[0081]

[0082] Among them, X best It is the current global optimal individual position. Let X be the updated position vector. i,j Let be the current position vector of the individual, t be the current iteration number, T be the maximum iteration number, rand be a uniformly distributed random number, and R be the exploration coefficient. τ is a perturbation factor, which can, to some extent, avoid interference caused by X. best The introduction of τ leads to a problem in the improved Pelican Search algorithm getting stuck in local optima during the search phase. The formula for calculating τ is as follows:

[0083]

[0084] In the formula, gamrnd() is the gamma random jump coefficient, which helps to escape local optima and enhances the diversity of τ values. Extensive experimental testing shows that the improved pelican optimization algorithm exhibits the strongest search capability and is more likely to escape local optima when the decision coefficient ζ = 0.6. By introducing a directional differential mutation strategy based on the decision coefficient, the convergence speed of the improved pelican optimization algorithm can be improved while reducing the risk of the algorithm getting trapped in local optima.

[0085] Step S104: Adaptive update of target scale through scale pyramid.

[0086] This invention draws on the design concept of mesoscale filters in the DSST algorithm, and applies it to the pelican individual X in a swarm intelligent target tracker. i =(x i ,y i ,c i By setting multiple target size variation ratios to generate samples with the same aspect ratio, a scale pyramid is constructed to predict scale variation, where x i Represents the x-coordinate and y-coordinate of individual i. i c represents the y-coordinate of individual i. i Let represent the aspect ratio of individual i. After determining the target coordinates in the current frame image, s scale parameters are selected centered on i to estimate its scale variation, resulting in s samples of different sizes. The formula for selecting the candidate region size is:

[0087]

[0088] Among them, M i and N i Let be the length and width of the target in the i-th frame image, respectively, and η be the scale factor, set to 1.02. s is the scale factor index, and s is the number of scale factors, which is set to 30.

[0089] As a preferred solution, when the target's angle changes, a fixed aspect ratio may cause the tracking system to gradually locate and focus only on local features of the target. Over time and with the accumulation of errors, this could lead to the tracker gradually losing sight of the target. To effectively adjust for changes in the target's aspect ratio, a two-dimensional scale perception factor c is designed. i =(c xi ,c yi ), where c xi and c yi These are the scale factors for the length and width of the target bounding box, respectively, used to reflect the perceived spatial scale in the x and y axes. The candidate region size selection formula is then updated to:

[0090]

[0091] Among them, c xi and c yi n represents the scale factor of the bounding box along the x-axis and y-axis, respectively. i ,n j An independent index for aspect ratio variation is provided to accommodate non-uniform scale variations of the target.

[0092] Figure 6 This is an illustration of scale variation under a fixed aspect ratio. During UAV flight missions, the distance between the target and the imaging device dynamically changes, causing fluctuations in the target size within the image. When the target's angle changes, a fixed aspect ratio may cause the tracking system to gradually locate and focus only on local features of the target. Over time and with the accumulation of errors, the tracker may gradually lose sight of the target.

[0093] Figure 7 This is an illustration of the scale-to-scale perception factor. To effectively adjust the aspect ratio of the target, a two-dimensional scale-to-scale perception factor c is designed. i =(c xi ,c yi ), where c xi and c yi These are the scale factors for the length and width of the target bounding box, respectively, used to reflect the perceived spatial scale in the x and y axes. Using the formula... Select a new candidate region size, n i and n j During the tracking process, changes do not necessarily occur simultaneously. Compared with target trackers that only have candidate boxes with fixed aspect ratios, it can obtain candidate boxes with varying aspect ratios when estimating the target size, thus adapting to changes in the target scale.

[0094] Step S105: Dynamically update the target template based on the adaptive learning rate.

[0095] The formula for calculating the adaptive learning rate is as follows:

[0096]

[0097] Where φ(t) is the learning rate of the feature template at frame t, φ max This is the peak learning rate, set to 0.05; f i (t) represents the fitness of the i-th individual, n is the population size; δ1 and δ2 are adjustment parameters, set to 10 and 5 respectively, which are used to adjust the fitness threshold for occlusion detection. The dynamic target feature template update strategy is as follows:

[0098] γ(t)=[1-φ(t)]γ(t-1)+φ(t)γ best (t)

[0099] Where γ(t) is the target feature template of the t-th frame, γ best (t) is the optimal target feature template for frame t.

[0100] γ best (t) represents the fusion feature corresponding to the optimal candidate target; through f i (t) controls the learning rate φ(t), and uses γ best (t) Update the target template to adaptively adjust the template based on the matching degree between the optimal candidate and the target template.

[0101] Figure 4 This is a target tracking framework based on a swarm intelligence algorithm. In the figure, the swarm intelligence algorithm population is denoted as X = {X1, X2, ..., X}. n Individuals X distributed within the search space i ={x i ,y i ,sx i ,sy i}, x i and y i sx and yx represent the x and y coordinates of the target's center position, respectively. i and sy i Let represent the scale factors of the target bounding box, and , where t represents time. During the tracking process, at frame t-1, the target's position and bounding box size are [x(t-1), y(t-1)] and [w(t-1), h(t-1)], respectively; at frame t, the target's position and bounding box size are [x(t), y(t)] and [sx...]. i ×w(t-1),sy i ×h(t-1)]. The swarm intelligence algorithm searches for candidate targets similar to those in previous frames within the search space of frame t based on a similarity function.

[0102] Figure 5 This document outlines the target tracking framework based on an improved Pelican Search algorithm. First, parameters are initialized and the first frame image is input to obtain initial target location and scale information. Then, a tracking loop begins: for each candidate location, multiple candidate boxes at different scales are generated. Then, fused features are extracted from each candidate box, and the Bhattacharyya similarity with the target template is calculated. The improved Pelican Search algorithm is used to simultaneously optimize the location and scale parameters, ultimately obtaining the optimal candidate target. Next, it is determined whether this is the last frame. If so, the target tracking result is output, and tracking ends; otherwise, dynamic feature template updates are performed, and target tracking begins in the next frame of the image.

[0103] Step S2: Design a long-term tracker based on the joint matching loss confidence, including determining the target tracking status based on the matching loss confidence and performing fast localization and re-detection when the target is lost. Each sub-step is described in detail below.

[0104] Step S201: Determine the target tracking status based on the confidence level of the matching loss.

[0105] The confidence score based on matching loss is calculated by comparing the predicted bounding box with the ground truth bounding box in the target tracking task to infer the confidence score. The classification loss uses the standard binary cross-entropy loss function to assess the deviation between the predicted and actual targets. The regression loss uses the generalized IoU loss function and the L1 norm loss function, where the IoU loss measures the tracking accuracy by calculating the ratio of the intersection to the union of the predicted and ground truth bounding boxes.

[0106] To comprehensively evaluate the confidence level of the tracker's matching loss, a comprehensive loss function L is defined. totle The calculation formula is as follows:

[0107] L totle =τ cls L cls +τ reg L reg

[0108] In the formula, L cls and L reg These are the classification loss function and the regression loss function, respectively, τ cls and τ reg These are the adjustment coefficients for classification loss and regression loss, respectively.

[0109] Due to its smooth properties, the Gaussian function can effectively capture subtle differences in confidence level as loss changes. Therefore, the Gaussian function is used to model tracking confidence level as follows:

[0110] C t =exp(-γ t L totle 2 )

[0111] In the formula, C t γ represents the tracking confidence in frame t. t To control the rate of confidence decrease, the parameter γ is adjusted. t The confidence level C can be flexibly controlled. t With loss L totle Sensitivity to change.

[0112] A target state judgment module is designed based on the confidence level of the matching loss, and the tracker switching factor is defined as Q. t (5) as follows:

[0113]

[0114] In the formula, θ is the set confidence threshold. A tracker switching factor of 0 indicates that the tracking state is reliable and the base tracker can stably track the target. If the confidence level C is 0 for 5 consecutive frames... t If all values ​​are below θ, the tracker switching factor is set to 1, indicating that the target is lost and the current base tracker status is unreliable. The search tracker needs to be activated to search for and retrieve the target.

[0115] The base tracker is responsible for tracking the target under normal conditions, estimating the target's position and motion state, and calculating the matching loss confidence score in each frame to determine the reliability of the tracking result. If the confidence score is lower than a set threshold for five consecutive frames, the tracking is considered to have failed and the target is lost. At this point, the search tracker is activated to quickly locate the target in the search area.

[0116] Step S202: Perform fast location re-detection when the target is lost.

[0117] Based on the target tracking status determination method in step S201, when the tracking confidence level is lower than a set threshold, the target tracking status is considered unreliable, and the search tracker can be invoked for rapid localization and re-detection to retrieve the target. The re-detection process includes:

[0118] a) Based on the Gaussian distribution-based short-term memory module, initialize the search population of the improved pelican search algorithm in the estimated possible target area. The population individuals contain the target's location and scale information.

[0119] b) For each candidate region in the search population, extract its multi-feature fusion representation and calculate the similarity with the target template as the fitness.

[0120] c) Using the improved pelican search algorithm, the population is iteratively updated based on the fitness to optimize the target's position and scale parameters until the optimal candidate target is obtained, thus completing the relocation and retrieval of the lost target.

[0121] Figure 8This is a flowchart of a long-term stable target tracking method for drone aerial photography. After reading the video sequence, a lightweight target tracking algorithm is first used as the base tracker. Specifically, after extracting target features from the template region and search region using a feature extraction network, the features are integrated through an attention-based feature fusion network to obtain the target's position. Based on this, regression and classification branches further process the fused features. The tracking state judgment module then evaluates the reliability of the current tracking state. If the target position obtained by the base tracker is reliable, tracking continues in the next frame. Conversely, if the current tracking state is deemed unreliable, i.e., the target may have been lost, an improved pelican search algorithm is immediately activated as a search tracker to relocate the target. After model initialization, CN features, FHOG features, and depth features are extracted, fused, and similarity calculated. Then, the improved pelican search algorithm is used to locate the target within the search region. After the search tracker obtains the target position and scale, the tracking state judgment module continues to make judgments. If the target has been found, the base tracker is reactivated; if the target has not been found, the next round of search is performed until the tracking task terminates.

[0122] Figure 9 The proposed Our algorithm and five comparative algorithms were quantitatively analyzed on the UAV123 dataset, resulting in overall accuracy and success rate graphs. To accurately evaluate and compare the performance differences between this algorithm and others, after completing the subjective qualitative analysis, quantitative analysis was further conducted using mathematical tools, resulting in accuracy and success rate curves for intuitive graphical representation. The accuracy curve measures the distance error between the tracking result and the ground truth annotation; the horizontal axis represents the threshold, and the vertical axis represents the algorithm's accuracy. The success rate curve evaluates the success rate of the tracking result within a certain error range; the horizontal axis represents the threshold, and the vertical axis represents the tracking success rate.

[0123] This invention was conducted on a computer with an Intel Core i7-12700H 2.60GHz CPU@2.60GHz, an NVIDIA GeForce RTX 3090 GPU, and 32GB of RAM. The operating system was Windows 10. The tracker was built based on Python 3.7 and PyTorch 1.4.0, the optimizer was Adam, the initial learning rate was 0.003, the weight decay was 0.01, the batch size was 4, and the epoch was 80.

[0124] The algorithm's parameter settings are shown in Table 1:

[0125] Table 1 Algorithm Parameter Settings

[0126]

[0127] The proposed Our algorithm and five comparative algorithms were quantitatively analyzed on the UAV123 dataset. The comparison of the obtained objective evaluation indicators is shown in Table 2.

[0128] Table 2 Comparison of Objective Evaluation Indicators for the Experiments of Six Algorithms

[0129]

[0130] The data in the table shows that Our algorithm has an accuracy of 0.703, a success rate of 0.808, and a running speed of 22.14 FPS, ranking first in overall performance among the compared algorithms. Specifically, Our algorithm improves accuracy by 2.18%, 1.74%, 35.19%, 60.14%, and 111.11% compared to the Pelican Optimization (POA), SSA, BES, MFO, and GWO algorithms, respectively. Its success rate is improved by 5.62%, 8.02%, 37.65%, 57.20%, and 132.85% compared to the same algorithms. The significantly improved running speed (FPS) of Our algorithm compared to the five compared algorithms is mainly due to the introduction of a Gaussian distribution-based short-term memory module. During initialization, this module concentrates the population around the target rather than distributing it uniformly across the image, avoiding ineffective iterations and improving optimization speed. Overall, compared with five other swarm intelligence-based tracking algorithms, Our algorithm shows improvements in both accuracy and success rate, and also exhibits the best real-time performance, making it well-suited for drone aerial target tracking scenarios.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit its scope of protection. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that after reading the present invention, they can still make various changes, modifications, or equivalent substitutions to the specific implementation of the invention, but all such changes, modifications, or equivalent substitutions are within the scope of protection of the pending claims. The present invention is applicable to long-term aerial photography scenarios using drones for continuous tracking of targets such as cars and ships, and can effectively cope with complex challenges such as obstruction, scale changes, and rapid movement, providing technical support for drone monitoring, aerial reconnaissance, and other applications.

Claims

1. A long-term target tracking method for UAV aerial photography based on matching loss confidence, characterized in that, Includes the following steps: The design of the target tracking framework includes multi-feature fusion extraction of the target object, calculation of the similarity between the target template and the candidate region, target search and matching using the improved pelican search algorithm, adaptive updating of the target scale through the scale pyramid, and dynamic updating of the target template based on the adaptive learning rate. Design a long-term tracker with joint matching loss confidence, including determining the target tracking status based on the matching loss confidence and performing fast localization and re-detection when the target is lost.

2. The UAV aerial long-term target tracking method based on matching loss confidence as described in claim 1, characterized in that, The multi-feature fusion extraction of the tracked target object includes: extracting CN features, FHOG features, and Pool4 layer features of the target region, and performing weighted fusion according to the following formula to obtain the fused multi-dimensional feature γ. t : c t =ω CN c CN +oh FHOG c FHOG +oh Pool4 c Pool4 Where, γ CN γ FHOG and γ Pool4 These represent the CN features, FHOG features, and Pool4 layer features of the target object, respectively, ω CN ω FHOG and ω Pool4 These are the feature fusion coefficients corresponding to different features; the fused multidimensional feature γ t Used to construct the feature model of the target template and the feature representation of the candidate region in the search phase.

3. The UAV aerial long-term target tracking method based on matching loss confidence as described in claim 1 or 2, characterized in that, The calculation of the similarity between the target template and the candidate region includes: calculating the similarity between the feature model p of the target template and the feature model q of the candidate region based on the Bhattacharyya coefficient, using the following formula: Where p = {p1, p2, ..., p} θ }, q={q1, q2, ..., q θ }, θ1 is the quantization level, p i and q i These represent the probabilities of the i-th feature appearing in the two feature models, respectively; the value of the Bhattacharyya coefficient B(p,q) serves as the fitness evaluation function of the improved pelican search algorithm in the target search matching process.

4. The UAV aerial photography long-term target tracking method based on matching loss confidence as described in claim 1, characterized in that, The target search matching using the improved pelican search algorithm includes: An improvement to the short-term memory module based on Gaussian distribution is made by replacing the initialization of the algorithm's initial population with a Gaussian distribution based on the prediction of the target's motion trajectory, instead of a uniform distribution. Based on the improved reproduction correction strategy of survival of the fittest, the reproduction range and number of offspring are dynamically determined according to the fitness of parent individuals, and the population is updated through secondary selection. Based on the improved fusion differential directional mutation strategy, a decision coefficient ζ and a determination coefficient c are introduced to dynamically select whether to perform mutation updates based on the current individual position or the global optimal position, so as to balance the algorithm's global exploration and local development capabilities.

5. The UAV aerial photography long-term target tracking method based on matching loss confidence as described in claim 4, characterized in that, The improvements to the Gaussian distribution-based short-term memory module include: Based on the optimal target position X obtained from the first two frames best(t-1) and X best(t-2) Calculate the mean μ of the population distribution in frame t. t =X best(t-1) +(X best(t-1) -X best(t-2) ), and based on this mean μ t and the covariance matrix χ of the population initialized in frame t t Construct a Gaussian distribution model to generate the initial population individual positions in frame t. This causes the initial population to concentrate in the area where the target may appear.

6. The UAV aerial photography long-term target tracking method based on matching loss confidence as described in claim 4, characterized in that, The improved position update rule based on the fusion differential directional mutation strategy is as follows: Where τ is the perturbation factor, generated by the gamma random jump coefficient, and X best It is the current global optimal individual position. Let X be the updated position vector. i,j Let t be the current position vector of the individual, t be the current iteration number, T be the maximum iteration number, rand be a uniformly distributed random number, and R be the exploration coefficient.

7. The UAV aerial photography long-term target tracking method based on matching loss confidence as described in claim 1, characterized in that, When achieving adaptive update of the target scale using a scale pyramid, the formula for selecting the candidate region size is as follows: Among them, M i and N i Let be the length and width of the target in the i-th frame image, respectively, and η be the scale factor, n s s is the scale factor index, and s is the number of scale factors. Multi-scale candidate regions with the same aspect ratio are constructed to achieve scale prediction. Additionally, a two-dimensional scale perception factor c is introduced. i =(c xi ,c yi The formula for selecting the candidate region size is updated to: (This is used to adjust the aspect ratio of the target bounding box.) Where c xi and c yi Here, n represents the scale factor of the bounding box along the x-axis and y-axis, respectively. i ,n j An independent index for aspect ratio changes.

8. The UAV aerial photography long-term target tracking method based on matching loss confidence as described in claim 1, characterized in that, The dynamically updated target template based on adaptive learning rate includes: The formula for calculating the adaptive learning rate is as follows: Where φ(t) is the learning rate of the feature template at frame t, φ max It is the peak learning rate, f i γ(t) represents the fitness of the i-th individual, n is the population size, and δ1 and δ2 are adjustment parameters; the target template update strategy is: γ(t) = [1-φ(t)]γ(t-1) + φ(t)γ best (t), where γ(t) is the target feature template of the t-th frame, γ best (t) is the optimal target feature template for frame t.

9. The UAV aerial photography long-term target tracking method based on matching loss confidence as described in claim 1, characterized in that, The determination of target tracking status based on matching loss confidence includes: Define the comprehensive loss function L totle =τ cls L cls +τ reg L reg L cls For classification loss, L reg For regression loss, τ cls and τ reg For adjustment coefficients; Based on the aforementioned comprehensive loss function L totle The tracking confidence C of frame t is obtained by modeling using a Gaussian function. t =exp(-γ t L totle 2 ), where γ t Parameters used to control the rate at which confidence levels decrease; Set a confidence threshold θ and define a tracker switching factor Q. t (5) If the tracking confidence C of 5 consecutive frames t If all values ​​are below the threshold θ, then Q t (5) =1, indicating the target is lost, and the search tracker is initiated to perform a re-detection process; otherwise, Q t (5) =0, indicating that the tracking status is reliable, and the basic tracker continues to execute the tracking task.

10. The UAV aerial photography long-term target tracking method based on matching loss confidence according to claim 4, characterized in that, The step of performing fast localization and re-detection when the target is lost includes: Based on a Gaussian distribution-based short-term memory module, a search population for the improved pelican search algorithm is initialized within the estimated possible target region. Individual population members contain the target's location and scale information. For each candidate region in the search population, its multi-feature fusion representation is extracted, and the similarity with the target template is calculated as the fitness. Using the improved pelican search algorithm, the population is iteratively updated based on the fitness, and the position and scale parameters of the target are optimized until the optimal candidate target is obtained, thus completing the relocation and retrieval of the lost target.

Images