Real-time target tracking method and system for unmanned aerial vehicle edge computing

Abstract

The application discloses a real-time target tracking method and system for unmanned aerial vehicle edge computing, wherein the method comprises the following steps: calculating the dynamic importance evaluation value of each target according to the dynamic information and task information of multiple targets; assigning different tracking computing resources and tracking strategies for different targets according to the dynamic importance evaluation value; generating an adjustment instruction for adjusting the flight attitude of the unmanned aerial vehicle based on the dynamic importance evaluation value, so that a high importance target is kept in a stable tracking field of view; and compensating and correcting the corresponding tracking prediction result according to the state change of the unmanned aerial vehicle caused by the adjustment instruction. The application effectively solves the balance problem of precision, efficiency and stability in multi-target tracking under the condition of limited resources.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) flight control technology, and in particular to a real-time target tracking method and system for UAV edge computing. Background Technology

[0002] With the integration of drones and edge computing technology, achieving real-time and robust multi-target tracking locally on the drone platform (edge ​​side) has become a key technical requirement in fields such as security inspection and emergency rescue. However, achieving efficient and stable multi-target tracking in an airborne edge computing environment faces a series of mutually restrictive technical challenges.

[0003] First, existing technologies mostly focus on optimizing the visual tracking algorithm itself, while neglecting the fact that as a moving platform, the UAV's active changes in flight attitude directly disturb the imaging field of view, thus affecting the stability of visual tracking. Traditional solutions typically treat the visual tracking module and the flight control module as independent or weakly coupled systems, causing the tracking algorithm to passively adapt to the platform's movement, making it extremely easy to lose track of the target when it is moving rapidly or performing complex maneuvers.

[0004] Secondly, edge computing devices are severely limited in terms of computing resources, memory, and power consumption. When faced with multiple targets, most existing methods allocate computing resources equally or with static priorities, failing to dynamically and finely schedule resources based on the actual threat level, behavioral intent, and task urgency of the targets. This results in either a failure to guarantee the tracking quality of critical targets due to the even distribution of resources in complex scenarios, or a loss of situational awareness over a wide area due to excessive resource concentration.

[0005] Furthermore, existing solutions lack a unified decision-making framework capable of assessing target importance and coordinating the allocation of computational resources and flight control accordingly. Vision, flight control, and resource management are often designed in isolation, making it difficult to form a closed-loop optimization. As a result, the overall system performance is limited by the weakest link, making it impossible to simultaneously achieve continuous and stable tracking of high-value targets and effective coverage of the global scenario under resource constraints. Summary of the Invention

[0006] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0007] To address the aforementioned technical problems, this invention provides the following technical solution: a real-time target tracking method for UAV edge computing, comprising the following steps:

[0008] Based on the dynamic information and task information of multiple objectives, calculate the dynamic importance evaluation value of each objective;

[0009] Based on the dynamic importance evaluation value, differentiated tracking computing resources and tracking strategies are allocated to different targets;

[0010] Based on the dynamic importance evaluation value, adjustment commands are generated to adjust the flight attitude of the UAV so that high-importance targets are kept within a stable tracking field of view;

[0011] Based on the changes in the UAV's state caused by the adjustment command, the corresponding tracking and prediction results are compensated and corrected.

[0012] As a preferred embodiment of the real-time target tracking method for UAV edge computing described in this invention, the calculation of the dynamic importance evaluation value includes calculation through a target importance evolution function, wherein the target importance evolution function integrates the target's dynamic attribute information, behavioral semantic information, and pre-set task association information.

[0013] As a preferred embodiment of the real-time target tracking method for UAV edge computing described in this invention, the dynamic attribute information includes, but is not limited to, the target's position, size, and motion vector in the image; the behavioral semantic information is extracted through a lightweight behavior recognition network; and the task association information is a pre-defined target identity weight related to the task script.

[0014] As a preferred embodiment of the real-time target tracking method for UAV edge computing described in this invention, the expression for the target importance evolution function is:

[0015]

[0016] in, For the goal exist The dynamic importance evaluation value at any given moment; This is a stability scoring function based on position, size, and motion vectors; This is a behavior scoring function based on the output of a behavior recognition network; Static task weights; , , The fusion weights are dynamically adjustable.

[0017] As a preferred embodiment of the real-time target tracking method for UAV edge computing described in this invention, wherein: the stability scoring function It is configured such that the output value is higher when the target is closer to the edge of the image or when its motion vector points further out of the image, and the increasing trend is non-linear.

[0018] The behavior scoring function It is configured to: calculate the deviation of the current behavior from the expected normal behavior based on the output of the lightweight behavior recognition network and combined with a pre-set behavior rule library corresponding to the target type, and use the deviation as the main output of the behavior scoring function.

[0019] As a preferred embodiment of the real-time target tracking method for UAV edge computing described in this invention, the method includes the following steps: Allocating differentiated tracking computing resources and tracking strategies to different targets.

[0020] Set a dynamic threshold, and classify targets whose current dynamic importance evaluation value is higher than the dynamic threshold into the first tracking set, and classify the remaining targets into the second tracking set;

[0021] For each target in the first tracking set, a first type of tracking resource is allocated, the first type of tracking resource including invoking a high-precision visual tracking algorithm and processing at a baseline frame rate;

[0022] For targets in the second tracking set, allocate a second type of tracking resources, the second type of tracking resources including invoking a lightweight motion predictor and performing state updates at a reduced frame rate below the baseline frame rate;

[0023] The dynamic threshold and the reduction frame rate ratio are adaptively adjusted based on the dispersion of the current dynamic importance evaluation values ​​of all targets.

[0024] As a preferred embodiment of the real-time target tracking method for UAV edge computing described in this invention, the adaptive adjustment includes calculating the variance of the current dynamic importance evaluation values ​​of all targets. The larger the variance, the higher the dynamic threshold and the lower the frame rate reduction ratio, so as to further concentrate computing resources on high-importance targets.

[0025] As a preferred embodiment of the real-time target tracking method for UAV edge computing described in this invention, the method includes the following steps: Based on the dynamic importance evaluation value, generating adjustment commands for adjusting the UAV's flight attitude includes the following steps:

[0026] Select the top N key targets based on their dynamic importance evaluation values;

[0027] Calculate the weighted center position of the key target in the image coordinate system and compare it with the preset ideal tracking point to obtain the position deviation vector;

[0028] Based on the position deviation vector, the adjustment command is generated by a nonlinear controller, and the adjustment command includes at least the adjustment amount of the yaw rate and pitch rate of the UAV.

[0029] As a preferred embodiment of the real-time target tracking method for UAV edge computing described in this invention, the method includes the following steps: compensating and correcting the corresponding tracking prediction results based on the UAV state changes caused by the adjustment command.

[0030] Real-time acquisition of the actual angular velocity and linear acceleration information of the UAV after the execution of the adjustment command;

[0031] The angular velocity and linear acceleration information are input into a motion compensation model, which is used to estimate the background motion of the image caused by the UAV’s own maneuvering.

[0032] The output of the motion compensation model is used to perform reverse compensation on the target position prediction generated by the tracking strategy to obtain the corrected final tracking position.

[0033] This invention also provides a real-time target tracking system for UAV edge computing, applied to the aforementioned real-time target tracking method for UAV edge computing, comprising:

[0034] Airborne edge computing collaboration module, deployed on the airborne edge computing device of the drone;

[0035] The flight control state coupling module is located in the flight control system of the UAV and is communicatively connected to the airborne edge computing collaboration module. It is used to send the UAV state change information caused by the adjustment command to the tracking result correction unit in real time.

[0036] The airborne edge computing collaboration module includes:

[0037] The resource scheduling and tracking execution unit is connected to the importance assessment unit and is used to allocate differentiated tracking computing resources and tracking strategies to different targets based on the dynamic importance assessment value, and generate tracking prediction results for each target;

[0038] The flight control command generation unit is connected to the importance assessment unit and the resource scheduling and tracking execution unit, and is used to generate adjustment commands for adjusting the flight attitude of the UAV based on the dynamic importance evaluation value.

[0039] The tracking result correction unit is connected to the resource scheduling and tracking execution unit and the flight control command generation unit, and is used to compensate and correct the corresponding tracking prediction results based on the UAV state changes caused by the adjustment command.

[0040] The beneficial effects of this invention are:

[0041] 1. This invention uses a unified dynamic importance evaluation value as the core of decision-making to form a closed-loop collaboration of "perception-evaluation-decision-control-compensation", which simultaneously realizes intelligent scheduling of edge computing resources and active control of UAV flight attitude. Under resource-constrained conditions, it effectively solves the problem of balancing accuracy, efficiency and stability in multi-target tracking.

[0042] 2. This invention calculates the variance of the dynamic importance evaluation values ​​of all targets in real time. Based on this variance, the dynamic threshold is increased and the frame rate is reduced, so that when the scene importance distribution is discrete (i.e., the variance is large), the system can automatically concentrate computing resources on high-importance targets. Thus, under the condition of strictly limited edge computing resources, the system achieves the ultimate guarantee of tracking accuracy and stability of core targets, while maintaining the baseline monitoring capability of non-core targets, and achieves the optimal balance between global resource utility and task performance.

[0043] 3. This invention selects the top N key targets with dynamic importance evaluation values, calculates their weighted center position and compares it with the ideal tracking point to obtain the position deviation vector, and then drives the nonlinear controller to directly generate the adjustment amount of the UAV's yaw rate and pitch rate. This enables the UAV's flight attitude to actively and accurately serve to stabilize the high-importance target within the stable tracking field of view, realizing a closed-loop mapping from visual semantic cognition to aircraft physical control, and breaking through the limitation of the tracking algorithm and flight control system being independent of each other in traditional methods. Attached Figure Description

[0044] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0045] Figure 1 This is an overall flowchart of the real-time target tracking method for UAV edge computing of the present invention.

[0046] Figure 2 This is a flowchart illustrating the dynamic importance evaluation value calculation process of the real-time target tracking method for UAV edge computing according to the present invention.

[0047] Figure 3 This is a flowchart illustrating the differentiated resource and strategy allocation process of the real-time target tracking method for UAV edge computing according to the present invention. Detailed Implementation

[0048] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0049] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0050] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0051] Secondly, the present invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of the present invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not according to the usual scale. Furthermore, the schematic diagrams are merely examples and should not limit the scope of protection of the present invention. In addition, actual fabrication should include three-dimensional spatial dimensions of length, width, and depth.

[0052] Example 1

[0053] Reference Figure 1-3 This is the first embodiment of the present invention, which provides a real-time target tracking method for UAV edge computing, including the following steps:

[0054] Before describing the specific steps of this invention, it is important to understand its application scenario: This real-time target tracking method for UAV edge computing is primarily active during the UAV's mission execution phase, including but not limited to the cruise search and lock-on tracking phases. When the UAV enters a phase not dominated by tracking tasks, such as return-to-home, takeoff, landing, or emergency abort, the system can control the cooperative tracking closed loop consisting of steps S1 to S4 to pause or enter a low-power standby state, and switch to the corresponding basic flight control mode. The following implementation methods are all described within this application context.

[0055] S1: Calculate the dynamic importance evaluation value of each objective based on the dynamic information of multiple objectives and task information.

[0056] Specifically, the dynamic importance evaluation value is calculated using a target importance evolution function, which integrates the target's dynamic attribute information, behavioral semantic information, and pre-defined task-related information. The dynamic attribute information includes, but is not limited to, the target's position, size, and motion vector in the image; the behavioral semantic information is extracted using a lightweight behavior recognition network; and the task-related information consists of pre-defined target identity weights associated with the task script.

[0057] Furthermore, the expression for the target importance evolution function is:

[0058]

[0059] in, For the goal exist The dynamic importance evaluation value at any given moment; This is a stability scoring function based on position, size, and motion vectors; This is a behavior scoring function based on the output of a behavior recognition network; Static task weights; , , The fusion weights are dynamically adjustable.

[0060] Among them, the stability scoring function Configured so that the output value is higher when the target's position in the image is closer to the edge, or when its motion vector points further out of the image, with a non-linear increasing trend; behavior scoring function. It is configured to: calculate the deviation of the current behavior from the expected normal behavior based on the output of the lightweight behavior recognition network and combined with a pre-set behavior rule library corresponding to the target type; and use the deviation as the output of the behavior scoring function. The mission script for this flight is pre-loaded and calibrated during system initialization. For example, in a key area patrol mission, the static weight of personnel targets can be set to 1, vehicles to 0.7, and animals to 0.2. The target identity (category) is provided by the aforementioned lightweight behavior recognition network, which ensures that the importance assessment is consistent with the top-level mission objectives.

[0061] In one specific implementation, the acquisition of dynamic attribute information, behavioral semantic information, and pre-defined task association information employs mature technologies suitable for edge computing. For example:

[0062] Dynamic attribute information is obtained in real time directly from the front-end output of the UAV visual tracking system, and its core data includes the target's position in the image. ),size( ) and motion vectors ( These data originate from the real-time processing results of airborne camera video by a lightweight target detection and tracking module (e.g., a YOLO-based lightweight detector combined with a Kalman filter) running on an edge computing device. This information directly reflects the stability of the target within the field of view and serves as the underlying basis for evaluation.

[0063] Behavioral semantic information is extracted through a lightweight behavior recognition network. This network takes as input a sequence of images of the target region (e.g., cropped blocks of the target region from 3-5 consecutive frames). The network structure can employ a depthwise separable convolution-optimized 3D CNN or a Two-Stream network, significantly reducing computational load while maintaining accuracy. The network output is either a behavior category probability vector or a feature vector representing high-level behavior. Its role is to provide a high-level semantic understanding of the target's intent and anomalies, enabling the system to distinguish the importance of the target from the level of task-related behaviors (such as identifying "loitering" and "going against the flow").

[0064] Pre-defined mission-related information consists of static configuration parameters pre-set based on the objectives of this flight mission (e.g., assigning different basic weights to different target types). Its purpose is to introduce prior mission guidance, ensure that the importance assessment is consistent with the top-level mission objectives, and ensure that the system prioritizes the types of key mission objectives.

[0065] In one specific implementation, the stability scoring function This is designed to quantify the risk of losing a target; a higher output value indicates lower stability and a greater need for urgent attention. The specific calculation method is as follows:

[0066] First, calculate the target's edge distance risk factor. :

[0067] ;in The shortest distance from the target to the four sides of the image. The length of the image diagonal. The sensitivity coefficient is greater than 0. This formula ensures that the closer the target is to the edge, the higher the sensitivity coefficient. The non-linear (exponential) value approaches 1.

[0068] Secondly, calculate the target's motion escape risk factor. : ,in The angle between the target motion vector and the vector pointing from the image center to the target's location. When the motion direction points outwards from the image ( When <90°, A value greater than 0 indicates an escape tendency.

[0069] Finally, the stability score is calculated comprehensively:

[0070] ;in, , , These are the weighting coefficients. The normalized target size, for example , The total area of ​​the image. This term reflects the tracking reliability risk caused by a reduction in the relative size of the target. The smaller the relative size, the larger the value of this term, indicating that the target is less salient in the image and more vulnerable to tracking, thus driving the stability score. The score increases accordingly. This function ensures that the score increases significantly and non-linearly when the target approaches the edge or moves outward, accurately triggering high-priority tracking.

[0071] It should be noted that in specific engineering implementations, the weighting coefficients... , , The settings are not fixed values, but are determined through a combination of system calibration and rule configuration, based on the mission type, the drone platform, and the physical characteristics of the sensors. The core principle lies in quantifying the contribution weight of different risk factors to the "probability of losing track of the target." A typical empirical initial configuration is as follows: .

[0072] In one specific implementation, the scoring function The purpose is to quantify the abnormality of target behavior, and its specific calculation method is as follows:

[0073] First, the lightweight behavior recognition network outputs behavior feature vectors or category probabilities.

[0074] Secondly, the system pre-defines a target type-behavior rule base. For example, for the "vehicle" type, the rule base might define: "driving at a constant speed" as normal behavior (deviation = 0), "driving against traffic" as high-risk abnormal behavior (deviation = +1.0), and "illegally parked" as medium-risk abnormal behavior (deviation = +0.6). For the "personnel" type, "walking" is defined as normal, "running" as possibly medium-risk, and "fighting" as high-risk.

[0075] Then, the scoring function Based on the current target identification type, the rule base is queried, and a rule mapping operation is performed: the currently identified behavior is matched with the behavior-deviation mapping table preset for this type of target in the rule base, thereby obtaining a preset deviation score. This score is directly used as... The output of this method is optimized. This design transforms behavior scoring from a complex probability comparison into an intuitive, rule-based deviation assessment based on domain knowledge. It is computationally efficient and highly interpretable, making it particularly suitable for edge computing scenarios.

[0076] In a preferred embodiment, the fusion weights , , The dynamic adjustment strategy adopts a two-layer approach, combining "preset switching based on task phase" and "adaptive fine-tuning based on real-time scenario situation."

[0077] Preset switching based on task phase: The system predefines several tracking task phases (e.g., cruise search phase, lock-on tracking phase) and configures a set of preferred weight preset values ​​for each phase. When the task enters different phases, the weights automatically switch to change the evaluation focus.

[0078] Adaptive fine-tuning based on real-time scene situation: At any given stage, the system monitors the scene's "attention distraction" in real time, for example, calculating the stability score of all targets. The variance. When the variance exceeds a preset threshold, it indicates that multiple targets are simultaneously at high risk of loss. At this stage, the system dynamically fine-tunes the weights based on the baseline to increase the risk. The weighting of the stability scoring function is adjusted so that its output is weighted in the final importance evaluation value. This allows the overall assessment results to more sensitively respond to and highlight those targets with the highest immediate risk of loss, providing clearer signals for accurate decision-making in subsequent steps.

[0079] It should be emphasized that the above-mentioned fusion weights , , The core of the dynamic adjustment strategy lies in establishing a closed-loop feedback mechanism from scenario situation awareness to weight mapping. The specific baseline value, adjustment range, and trigger threshold of the weights can be determined by those skilled in the art based on actual task requirements and platform characteristics through conventional system calibration methods.

[0080] It should be noted that step S1 transforms the traditional, single, static target priority judgment into a multi-dimensional, interpretable, and adaptable dynamic cognitive process that can adapt to changes in tasks and scenarios by constructing a "target importance evolution function" that integrates the target's underlying motion stability, high-level behavioral semantics, and top-level task priors, and by designing specific calculation rules and dynamic weight adjustment mechanisms for the key scoring functions within it. This provides a precise decision-making basis for subsequent steps such as "allocating differentiated tracking computing resources and tracking strategies to different targets" and "generating adjustment commands for adjusting the UAV's flight attitude or motion state".

[0081] S2: Based on the dynamic importance evaluation value, allocate differentiated tracking computing resources and tracking strategies to different targets.

[0082] Specifically, allocating differentiated tracking computing resources and tracking strategies to different targets includes the following steps:

[0083] S21: Set a dynamic threshold, and classify targets whose current dynamic importance evaluation value is higher than the dynamic threshold into the first tracking set, and classify the remaining targets into the second tracking set;

[0084] S22: For each target in the first tracking set, allocate a first type of tracking resource, which includes calling a high-precision visual tracking algorithm and processing at a baseline frame rate;

[0085] S23: For the target in the second tracking set, allocate a second type of tracking resources, which includes calling a lightweight motion predictor and updating the state at a reduced frame rate below the baseline frame rate;

[0086] The dynamic threshold and the proportion of reduced frame rate are adaptively adjusted based on the dispersion of the current dynamic importance evaluation values ​​of all targets.

[0087] Furthermore, the adaptive adjustment specifically includes:

[0088] Calculate the variance of the dynamic importance evaluation values ​​for all current targets. The larger the variance, the higher the dynamic threshold and the lower the frame rate reduction ratio, so as to further concentrate computing resources on high-importance targets.

[0089] In a general implementation, step S2, through the aforementioned differentiated allocation, ultimately generates a continuously updated "tracking prediction result" for each tracked target. This result will serve as the direct basis for flight decisions and status corrections in subsequent steps (S3) and (S4). The generation method and reliability of the "tracking prediction result" differ for different sets of targets, as detailed below.

[0090] In one specific implementation, the first type of tracking resources is implemented by allocating high-fidelity, high-response tracking resources and strategies to high-importance targets classified into the first tracking set. Specific implementation includes:

[0091] At the algorithm level: A high-precision visual tracker is instantiated for each target. This high-precision visual tracker can employ existing algorithms such as discriminative correlation filtering (e.g., BACF) or lightweight Siamese networks (e.g., a mobile-optimized version of SiamRPN++), maintaining high tracking accuracy even in complex backgrounds and with partial occlusion. Furthermore, after each execution, the high-precision visual tracking algorithm outputs high-confidence target state information, including its precise location, bounding box, and motion velocity in the image; this output constitutes the tracking prediction result for that target.

[0092] At the computational resource level: Allocate a dedicated or high-priority computation thread to each high-precision visual tracker to ensure that its computational needs can be responded to in a timely manner and avoid tracking delays or failures due to resource contention.

[0093] At the data processing level: The input video stream is processed using a baseline frame rate (such as 30 FPS or the camera's full frame rate). This means that for each high-importance target, the system performs a complete tracking forward inference in every frame (or every N frames, where N is small) to obtain its latest and most accurate positional state.

[0094] In one specific implementation, the second type of tracking resources is implemented by allocating high-efficiency, low-overhead predictive tracking resources to low-importance targets classified into the second tracking set. Specific implementation includes:

[0095] At the algorithm level: A lightweight motion predictor is used instead of a complete visual tracker. For example, a simplified motion model (such as a uniform velocity (CV) model or a uniform acceleration (CA) model) is maintained for each target, and its state updates involve only lightweight matrix operations, or a very simple Kalman filter is used. In addition, this lightweight motion predictor calculates the estimated position of the target based on the model in each update cycle, and this estimated position constitutes the tracking prediction result for that target.

[0096] At the computational resource level: motion predictors for all low-importance targets share one or more low-priority background computation threads. Their computational tasks can be processed in batches, thereby significantly reducing context switching and scheduling overhead.

[0097] At the data processing level: a reduced frame rate (e.g., 5-10 FPS) is used for state updates. The system only performs motion prediction (calculating the new position based on the model) for such targets at the frame time corresponding to the reduced frame rate, while directly using the prediction results from the previous cycle in other frames, thereby saving a lot of unnecessary visual computation.

[0098] In one specific implementation, the differentiated tracking strategy is manifested in the use of fundamentally different state estimation and maintenance logics for targets in the first tracking set and the second tracking set:

[0099] For the first tracking set (high-importance targets), the tracking strategy adopts an "active perception, precise correction" approach. Its core relies on a high-precision visual tracking algorithm, performing a complete target appearance matching and position regression based on newly captured image data in each processing cycle. This strategy aims to obtain absolute coordinate updates and can effectively handle changes in target appearance, scale, and complex maneuvers, but it incurs significant computational overhead.

[0100] For the second tracking set (low-importance targets), the tracking strategy adopts a "motion modeling and prediction-based" approach. Its core relies on a lightweight motion predictor, using the target state confirmed in the previous cycle as a starting point, and calculating its predicted position in the current frame based on a pre-defined motion model (such as a constant velocity model). This strategy can only be used for simple correlation verification or model correction with a small amount of image information at lower frame rate reduction times. Its advantage is extremely low computational cost, but it cannot handle sudden maneuvers or drastic changes in the target's appearance.

[0101] Strategy linkage and escalation / de-escalation: When the motion prediction error of a target in the second tracking set exceeds a preset error tolerance threshold, or its dynamic importance evaluation value... When the dynamic importance evaluation value of a target in the first tracking set continues to rise due to changes in behavior, the system can trigger a policy upgrade, temporarily assigning it to the first type of resource and performing visual verification. Once verification is successful, it can be assigned to the first tracking set. Conversely, if the dynamic importance evaluation value of a target in the first tracking set... If the value remains below the dynamic threshold, a policy downgrade is triggered, and the tracking policy can be seamlessly switched to the policy of the second tracking set to free up resources.

[0102] In a more specific implementation, the operation of temporarily allocating the first type of resources and performing visual verification is managed by a high-priority interrupt service routine in the edge computing collaborative processing module or a separate monitoring thread. When the upgrade conditions are triggered, this management mechanism performs the following actions:

[0103] Resource borrowing: The computing resources and time slots required for a high-precision visual tracking can be "borrowed" from the dynamic resource pool reserved by the system, or by temporarily suspending a single calculation cycle of a target that is currently in the "first tracking set" but has a very high confidence level.

[0104] Verification execution: Using borrowed resources, perform a complete high-precision visual tracking operation on the target that needs to be upgraded.

[0105] Decision and Resource Release / Fixation: If the verification is successful (i.e., high confidence in visual tracking and position consistent with prediction), the target is officially moved to the first tracking set and permanently assigned the first type of tracking resources. Temporary resources are released or the suspended task is resumed. If the verification fails, temporary resources are released, the target remains in the second set, but its motion model may be reset based on the verification results.

[0106] This mechanism ensures the flexibility and real-time nature of resource adjustments, while avoiding system performance fluctuations caused by frequent resource reallocation.

[0107] The error tolerance threshold is used to determine whether the "motion modeling and prediction-based" strategy currently employed by the second tracking set is still effective. Its value is usually preset based on the target type, sensor noise characteristics, and acceptable tracking error range, and is a relatively stable engineering parameter.

[0108] In one specific implementation, the distribution dispersion (i.e., variance) of the target dynamic importance evaluation values ​​is considered. The dynamic threshold and frame rate reduction ratio are dynamically adjusted. The specific implementation process is as follows: through variance... Functional relationship with dynamic threshold and variance The updated dynamic threshold and the percentage of frame rate reduction are calculated by relating the dynamic threshold to the percentage of frame rate reduction.

[0109] In a preferred embodiment, variance The functional relationship between the dynamic threshold and the dynamic threshold is expressed as follows: ,in, The basic threshold can be preset according to the task stage; This is a direct proportionality coefficient. This expression indicates that as the difference in importance between objectives increases, i.e., Increasing the threshold means that there are both extremely high and extremely low importance objectives in the scene. The system automatically raises the dynamic threshold to filter more objectives into the second tracking set, ensuring that the first tracking set contains only the most critical objectives, thus achieving extreme resource focus.

[0110] In a preferred embodiment, variance The functional relationship between the reduction in frame rate and the ratio is expressed as follows: , where 0 < <1, This is the minimum allowable percentage limit to prevent tracking from completely stalling. The basic proportion; It is a direct proportionality coefficient. This expression indicates that when... When the frame rate is increased, the system substantially reduces the frequency of image processing and state updates for targets in the second tracking set by decreasing the reduction ratio. This directly reduces the frame processing time overhead for low-importance targets, thereby allowing the saved edge computing system time budget and computing power to be more concentrated on high-importance targets in the first tracking set, thus strengthening differentiated control at the temporal resource allocation level.

[0111] It should be noted that step S2 uses a variance of the target's dynamic importance evaluation value. To facilitate the adaptive adjustment of feedback signals, the following operations are dynamically performed: First and second tracking sets are divided based on dynamic thresholds; first-type tracking resources are allocated to targets in the first tracking set using a "proactive perception, precise correction" strategy; second-type tracking resources are allocated to targets in the second tracking set using a "motion modeling, prediction-based" strategy. This step transforms the dynamic importance evaluation value output from S1 into a differentiated scheduling scheme for edge computing resources and generates continuously updated tracking prediction results for each target. This achieves a balance between tracking accuracy and resource consumption at the system level, providing hierarchical and reliable input data for subsequent S3 (adjustment instructions) and S4 (compensation correction) steps.

[0112] S3: Based on the dynamic importance evaluation value, generate adjustment commands to adjust the UAV's flight attitude so that high-importance targets remain within a stable tracking field of view.

[0113] It should be noted that step S3 aims to combine the "dynamic importance evaluation value" calculated in step S1 with the "tracking prediction result" output in step S2 to generate precise commands that directly drive the UAV's flight actions. Its implementation revolves around a core principle: enabling the UAV's flight attitude to actively serve to maintain the optimal tracking position of high-importance targets within the image's field of view.

[0114] Specifically, based on the dynamic importance evaluation value, adjustment commands are generated to regulate the flight attitude of the UAV, including the following steps:

[0115] S31: Select the top N key targets based on their dynamic importance evaluation values;

[0116] S32: Calculate the weighted center position of the key target in the image coordinate system and compare it with the preset ideal tracking point to obtain the position deviation vector;

[0117] S33: Based on the position deviation vector, the adjustment command is generated by a nonlinear controller, and the adjustment command includes at least the adjustment amount of the yaw rate and pitch rate of the UAV.

[0118] In one specific implementation, the value of N is not fixed, but is dynamically determined based on the current task stage and computational load.

[0119] In a preferred embodiment, the value of N can be dynamically determined. Preferably, an automatic determination method based on importance accumulation is used: an importance accumulation threshold is set, for example, 0.7, i.e., 70%. The system starts with the target with the highest evaluation value and accumulates its normalized importance until the accumulated sum reaches or exceeds the importance accumulation threshold. The number of targets participating in the accumulation at this point is the current N. This method ensures that the generation of flight control commands always focuses on the core subset of targets that contribute the vast majority of dynamic importance in the scenario, avoiding flight command jitter or control focus dispersion caused by tracking too many targets.

[0120] In one specific implementation, in step S32, the weighted center position of the key target in the image coordinate system will be used as the desired tracking point that the UAV camera's line of sight should point to. Specifically, let the two-dimensional pixel coordinates of each key target in the image be... Then the coordinates of the expected tracking point Calculated using the following formula:

[0121] Horizontal coordinate: ;

[0122] Vertical coordinates: ;

[0123] The numerator of the formula weights the target positions according to their dynamic importance, while the denominator is normalized, thus yielding a stable weighted average position that reflects the distribution of the relative importance of each target.

[0124] In one specific implementation, the ideal tracking point preset in step S32, for example, is... This is typically set to the center point of the image, or a preset offset point based on the task.

[0125] In one specific implementation, in step S32, the position deviation vector, for example, is... Let the position deviation vector be represented, then - , is a two-dimensional vector.

[0126] In one specific implementation, step S33, the adjustment instruction generation process includes the following steps:

[0127] Nonlinear mapping and processing: The controller's response to the input position deviation vector Nonlinear processing is characterized by the following features:

[0128] Dead zone: When the positional deviation is within a certain range When the pixel value is less than the preset pixel threshold, the controller output is zero, which avoids unnecessary frequent fine-tuning of the drone caused by image noise or slight jitter, thus improving flight stability.

[0129] Variable gain control: The controller's response gain (such as the proportional gain) can be configured to vary with the input. Increase the size of the target (e.g., a piecewise function). This allows the controller to react more quickly and violently when a high-importance target deviates significantly from its ideal position, enabling rapid callback; while when the deviation is small, fine-tuning is performed to achieve smooth and stable tracking.

[0130] Output limiting: The final output angular velocity command value is strictly limited to the maximum angular velocity range that is safely allowed by the UAV platform, i.e. .

[0131] Command calculation: The output of the nonlinear controller is the desired yaw and pitch angular velocities in the UAV body coordinate system after the above processing. (From the image position deviation vector) The conversion to angular velocity commands is achieved through a coordinate transformation model that integrates camera intrinsic parameters and the UAV-camera mounting relationship. This is a conventional technique for visual servo tracking in this field. These two angular velocity commands constitute the core adjustment commands sent to the UAV flight controller.

[0132] In a preferred embodiment, the nonlinear controller may employ a model predictive control (MPC) framework to enhance the smoothness and predictability of the system in dynamic scenarios. This framework performs roll optimization in each control cycle with the objective of minimizing the predicted position deviation vector ΔP over a finite future time domain while satisfying the UAV's dynamic constraints. It directly solves for the optimal angular velocity command sequence and outputs the first command. This method can better handle control constraints and, to some extent, predict the target's motion trend, achieving smoother and more accurate tracking guidance.

[0133] It should be noted that step S3 above generates instructions to directly adjust the yaw and pitch angular velocities of the UAV by screening key targets, calculating the weighted center position deviation vector and processing it through a nonlinear controller. This enables the flight attitude to actively serve to stabilize high-importance targets within the preset tracking field of view, thereby realizing a closed-loop mapping and precise response from visual tracking task requirements to flight control actions, and providing a clear execution basis for the tracking correction in step S4.

[0134] S4: Based on the changes in the UAV state caused by the adjustment command, compensate and correct the corresponding tracking prediction results to form a tracking-flight control closed loop.

[0135] The purpose of this step is to eliminate the interference caused by the UAV's own maneuvering due to the adjustment commands in step S3, which affects the visual tracking results. This interference manifests as follows: when the UAV actively rotates or translates to track the target, the camera's field of view also moves, causing the stationary background to appear as if it is flowing in the image. If this motion is not compensated for, it will be incorrectly superimposed on the target's "tracking prediction result," causing a drift in the target's position estimate. Step S4 achieves accurate compensation through the following process:

[0136] S41: Real-time acquisition of the actual angular velocity and linear acceleration information of the UAV after the execution of the adjustment command;

[0137] S42: Input the angular velocity and linear acceleration information into a motion compensation model, which is used to estimate the background motion of the image caused by the UAV's own maneuvering.

[0138] S43: Using the output of the motion compensation model, perform reverse compensation on the target position prediction generated by the tracking strategy in step S2 to obtain the corrected final tracking position.

[0139] In one specific implementation, step S41 is executed concurrently with step 3. Simultaneously with the adjustment command being sent to the UAV flight controller (flight controller), a collaborative program located on the onboard edge computing device subscribes to and acquires raw data from the UAV's inertial measurement unit or the calculated state information from the flight control system in real time via a high-speed bus (such as CAN or SPI). This information includes at least the three-axis angular velocities and three-axis accelerations in the body coordinate system. This data is acquired synchronously with image frames or at a higher frequency, providing highly timely input for subsequent compensation.

[0140] In one specific implementation, the core of the motion compensation model is based on rigid body kinematics and camera perspective projection principles to calculate the theoretical velocity field (i.e., optical flow field) of each pixel in the image plane caused by the instantaneous motion of the UAV body. A specific model implementation is as follows:

[0141] Coordinate transformation: Transform the IMU data from the aircraft to the camera coordinate system. Let the rotation matrix between the camera and the aircraft be... The translation vector is (Given calibration parameters). Then, the angular velocity in the camera coordinate system... and linear velocity (The approximate instantaneous value, obtained by integrating acceleration or combining it with other state estimates, can be calculated using the following formula:) ,in The output speed can be set to a time interval, or by using a more accurate state estimator.

[0142] Image motion (optical flow) calculation: For any normalized pixel coordinate point (x, y) in the image (corresponding to the direction vector in the camera coordinate system), its motion velocity on the image plane. It can be approximately estimated by the following model: ,in, The image Jacobian matrix, whose elements are derived from camera intrinsic parameters (focal length) The depth information Z at that point (which can be assumed to be the average ground height or a rough estimate obtained through sensor fusion) determines the location. This represents the pixel movement speed (unit: pixels / second) caused by the drone's own maneuvering along the horizontal axis of the image. Positive values ​​usually indicate that the background is moving to the right, while negative values ​​indicate that it is moving to the left. This represents the pixel motion speed (in pixels per second) caused by the drone's own movement along the vertical coordinate axis of the image. Positive values ​​typically indicate downward background movement, while negative values ​​indicate upward movement. The model outputs... This represents the speed of background pixel movement at point (x, y) caused by the drone's own maneuvering.

[0143] In one specific implementation, the target position prediction generated by the tracking strategy in step S2 is reversed and compensated, including the following steps:

[0144] Obtain the original prediction: For each tracked target, based on the tracking set it was assigned to in step S2, obtain its original tracking prediction position in the current frame from the corresponding tracker or predictor assigned to it. Specifically, for targets in the first tracking set, the data is obtained from the output of their high-precision visual tracker; for targets in the second tracking set, the data is obtained from the output of their lightweight motion predictor.

[0145] Calculate the compensation amount: Substituting the coordinates into the motion compensation model above, the background velocity caused by the UAV maneuver at that point is calculated. Furthermore, based on the inter-frame time difference... Calculate the position offset vector of the point from the previous frame to the current frame due to the drone's movement. .

[0146] Inverse compensation: To obtain the target's motion relative to the real world, the "apparent motion" component caused by the UAV's own motion needs to be subtracted from the original observations. Therefore, inverse compensation is performed to obtain the corrected final tracking position. : The essence of this operation is to separate the image motion represented by the background points that are originally stationary in the world coordinate system from the image observation of the target, thereby obtaining a purer position estimate that reflects the true relative motion of the target.

[0147] Output and closed loop: The corrected This serves as the final, reliable tracking position output for the target in the current frame. This result is not only used for displaying or reporting the current tracking data, but more importantly, it will serve as a key input for steps S1 (dynamic importance assessment) and S2 (tracker update) in the next processing cycle. Thus, the state changes triggered by the proactive adjustment in S3, through precise compensation in S4, are fed back and integrated into the visual tracking loop, forming a complete and stable collaborative tracking closed loop from "perception-decision-control" to "perception correction".

[0148] In a preferred embodiment, the motion compensation model can be extended to a lightweight extended Kalman filter (EKF), which improves compensation accuracy and robustness. This filter takes the UAV's angular velocity and linear acceleration as input, and simultaneously estimates its own motion state (velocity and attitude changes) and inverse depth of key points in the camera coordinate system. Through the filter's prediction and update process, the motion field of the image background can be estimated more smoothly and accurately, especially when depth information is uncertain or sensor noise is high, exhibiting superior performance compared to directly calculated models.

[0149] In summary, this invention uses a unified dynamic importance evaluation value as the core of decision-making, forming a closed-loop collaboration of "perception-evaluation-decision-control-compensation," simultaneously achieving intelligent scheduling of edge computing resources and active control of UAV flight attitude. This effectively solves the challenge of balancing accuracy, efficiency, and stability in multi-target tracking under resource-constrained conditions. This invention calculates the variance of the dynamic importance evaluation values ​​of all targets in real time. Based on this variance, the dynamic threshold is increased and the frame rate is decreased by a certain ratio. This allows the system to automatically concentrate computing resources on high-importance targets when the scene importance distribution is discrete (i.e., large variance). Thus, under the condition of strictly limited edge computing resources, the system achieves the ultimate guarantee of tracking accuracy and stability of core targets, while maintaining baseline monitoring capability for non-core targets, achieving the optimal balance between global resource utility and task performance. This invention selects the top N key targets with dynamic importance evaluation values, calculates their weighted center position and compares it with the ideal tracking point to obtain the position deviation vector. This then drives the nonlinear controller to directly generate the adjustment amount of the UAV's yaw and pitch angular velocities. This enables the UAV's flight attitude to actively and accurately serve to stabilize high-importance targets within the stable tracking field of view, realizing a closed-loop mapping from visual semantic cognition to aircraft physical control, breaking through the limitation of the independent tracking algorithm and flight control system in traditional methods. This invention constructs a target importance evolution function that integrates dynamic attribute information such as the target's position, size, and motion vector in the image, behavioral semantic information extracted by a lightweight behavior recognition network, and pre-labeled task association information. It also configures specific response rules for the stability scoring function and the behavior scoring function, thereby achieving a multi-dimensional and interpretable dynamic evaluation of the target's importance. This provides a precise decision-making basis for subsequent differentiated resource allocation and flight adjustment.

[0150] Example 2: This example provides a real-time target tracking system for UAV edge computing, applied to the above tracking method, including:

[0151] Airborne edge computing collaboration module, deployed on the airborne edge computing device of the drone;

[0152] The flight control state coupling module is located in the flight control system of the UAV and is communicatively connected to the airborne edge computing collaboration module. It is used to send the UAV state change information caused by the adjustment command to the tracking result correction unit in real time.

[0153] The airborne edge computing collaboration module includes:

[0154] The resource scheduling and tracking execution unit is connected to the importance assessment unit and is used to allocate differentiated tracking computing resources and tracking strategies to different targets based on the dynamic importance assessment value, and generate tracking prediction results for each target;

[0155] The flight control command generation unit is connected to the importance assessment unit and the resource scheduling and tracking execution unit, and is used to generate adjustment commands for adjusting the flight attitude of the UAV based on the dynamic importance evaluation value.

[0156] The tracking result correction unit is connected to the resource scheduling and tracking execution unit and the flight control command generation unit, and is used to compensate and correct the corresponding tracking prediction results based on the UAV state changes caused by the adjustment command.

[0157] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A real-time target tracking method for edge computing of unmanned aerial vehicles, characterized in that, Includes the following steps: Based on the dynamic information and task information of multiple objectives, calculate the dynamic importance evaluation value of each objective; Based on the dynamic importance evaluation value, differentiated tracking computing resources and tracking strategies are allocated to different targets; Based on the dynamic importance evaluation value, adjustment commands are generated to adjust the flight attitude of the UAV so that high-importance targets are kept within a stable tracking field of view; Based on the changes in the UAV's state caused by the adjustment command, the corresponding tracking and prediction results are compensated and corrected.

2. The real-time target tracking method for UAV edge computing as described in claim 1, characterized in that: The calculation of the dynamic importance evaluation value includes calculation through a target importance evolution function, which integrates the target's dynamic attribute information, behavioral semantic information, and pre-set task association information.

3. The real-time target tracking method for UAV edge computing as described in claim 2, characterized in that: The dynamic attribute information includes, but is not limited to, the target's position, size, and motion vector in the image; the behavioral semantic information is extracted through a lightweight behavior recognition network; and the task association information is a pre-defined target identity weight related to the task script.

4. The real-time target tracking method for UAV edge computing as described in claim 3, characterized in that: The expression for the target importance evolution function is: in, For the goal exist The dynamic importance evaluation value at any given moment; This is a stability scoring function based on position, size, and motion vectors; This is a behavior scoring function based on the output of a behavior recognition network; Static task weights; , , The fusion weights are dynamically adjustable.

5. The real-time target tracking method for UAV edge computing as described in claim 4, characterized in that: The stability scoring function It is configured such that the output value is higher when the target is closer to the edge of the image or when its motion vector points further out of the image, and the increasing trend is non-linear. The behavior scoring function It is configured to: calculate the deviation of the current behavior from the expected normal behavior based on the output of the lightweight behavior recognition network and combined with a pre-set behavior rule library corresponding to the target type, and use the deviation as the main output of the behavior scoring function.

6. The real-time target tracking method for UAV edge computing as described in claim 1, characterized in that: Allocating differentiated tracking computing resources and tracking strategies to different targets includes the following steps: Set a dynamic threshold, and classify targets whose current dynamic importance evaluation value is higher than the dynamic threshold into the first tracking set, and classify the remaining targets into the second tracking set; For each target in the first tracking set, a first type of tracking resource is allocated, the first type of tracking resource including invoking a high-precision visual tracking algorithm and processing at a baseline frame rate; For targets in the second tracking set, allocate a second type of tracking resources, the second type of tracking resources including invoking a lightweight motion predictor and performing state updates at a reduced frame rate below the baseline frame rate; The dynamic threshold and the reduction frame rate ratio are adaptively adjusted based on the dispersion of the current dynamic importance evaluation values ​​of all targets.

7. The real-time target tracking method for UAV edge computing as described in claim 6, characterized in that: The adaptive adjustment includes calculating the variance of the current dynamic importance evaluation values ​​of all targets. The larger the variance, the higher the dynamic threshold and the lower the frame rate reduction ratio, so as to further concentrate computing resources on high-importance targets.

8. The real-time target tracking method for UAV edge computing as described in claim 1, characterized in that: Based on the dynamic importance evaluation value, adjustment commands for adjusting the flight attitude of the UAV are generated, including the following steps: Select the top N key targets based on their dynamic importance evaluation values; Calculate the weighted center position of the key target in the image coordinate system and compare it with the preset ideal tracking point to obtain the position deviation vector; Based on the position deviation vector, the adjustment command is generated by a nonlinear controller, and the adjustment command includes at least the adjustment amount of the yaw rate and pitch rate of the UAV.

9. The real-time target tracking method for UAV edge computing as described in claim 1, characterized in that: Based on the changes in the UAV's state caused by the adjustment command, the corresponding tracking and prediction results are compensated and corrected, including the following steps: Real-time acquisition of the actual angular velocity and linear acceleration information of the UAV after the execution of the adjustment command; The angular velocity and linear acceleration information are input into a motion compensation model, which is used to estimate the background motion of the image caused by the UAV’s own maneuvering. The output of the motion compensation model is used to perform reverse compensation on the target position prediction generated by the tracking strategy to obtain the corrected final tracking position.

10. A real-time target tracking system for UAV edge computing, applied to the real-time target tracking method for UAV edge computing as described in any one of claims 1-9, characterized in that, include: Airborne edge computing collaboration module, deployed on the airborne edge computing device of the drone; The flight control state coupling module is located in the flight control system of the UAV and is communicatively connected to the airborne edge computing collaboration module. It is used to send the UAV state change information caused by the adjustment command to the tracking result correction unit in real time. The airborne edge computing collaboration module includes: The resource scheduling and tracking execution unit is connected to the importance assessment unit and is used to allocate differentiated tracking computing resources and tracking strategies to different targets based on the dynamic importance assessment value, and generate tracking prediction results for each target; The flight control command generation unit is connected to the importance assessment unit and the resource scheduling and tracking execution unit, and is used to generate adjustment commands for adjusting the flight attitude of the UAV based on the dynamic importance evaluation value. The tracking result correction unit is connected to the resource scheduling and tracking execution unit and the flight control command generation unit, and is used to compensate and correct the corresponding tracking prediction results based on the UAV state changes caused by the adjustment command.

Images