Unmanned aerial vehicle target tracking method, device, equipment, medium and product

Abstract

The invention discloses an unmanned aerial vehicle target tracking method, device, equipment, medium and product, and relates to the technical field of unmanned aerial vehicle autonomous flight and intelligent control, and the method comprises the steps: obtaining the observation data of an airborne sensor, building an environment map, and obtaining a prediction position sequence of a target through a state estimation method; taking all reachable points corresponding to each prediction position as candidate sub-target points to carry out segmented path planning to obtain relay points; constructing an initial seed according to the initial position and the initial speed direction of the unmanned aerial vehicle, and generating a safe flight corridor by adopting an iterative region expansion method; obtaining a trajectory constraint by adopting a perception constraint method; and based on the safe flight corridor and the trajectory constraint, adopting a trajectory optimization method to obtain a tracking trajectory, and enabling the unmanned aerial vehicle to track the target along the tracking trajectory. According to the invention, through target position prediction, segmented path planning and rolling safe flight corridor generation, the real-time performance, safety and robustness of the unmanned aerial vehicle for moving target tracking in a dynamic environment are improved.

Description

Technical Field

[0001] This application relates to the field of autonomous flight and intelligent control technology for unmanned aerial vehicles (UAVs), and in particular to a UAV target tracking method, device, equipment, medium, and product. Background Technology

[0002] In recent years, the demand for drone target tracking has grown rapidly for applications such as security patrol, emergency rescue, smart cities, and aerial filming. Unlike traditional fixed-point trajectory planning, target tracking requires drones to generate spatiotemporal trajectories in real time in dynamic environments. This requires ensuring continuous visibility of the target and an appropriate line-of-sight distance, while also meeting the requirements for safe avoidance of static / dynamic obstacles and dynamic feasibility constraints such as platform speed, acceleration, and jerk. With the development of airborne sensors and computing platforms, online perception based on vision and multi-sensor fusion is gradually becoming mainstream. However, complex occlusion, multi-source interference, and sudden target maneuvers still pose significant challenges to drone target tracking.

[0003] Existing technologies mainly fall into three categories: First, tracking trajectory generation methods based on graph search or sampling, such as... Algorithms and Rapidly-exploring Random Tree (RRT) methods are common approaches. While simple to implement, their fragmented "path first, smooth later" approach can easily compromise visibility and line-of-sight constraints. Furthermore, frequent target maneuvers necessitate repeated replanning, leading to accumulated time delays. Secondly, rolling optimization methods based on Model Predictive Control (MPC) significantly increase problem dimensionality and nonlinearity when introducing non-convex constraints such as obstacle avoidance, unobstructed views, and distance preservation. Solving these methods relies on linearization and soft constraint penalties, making them prone to local optima or infeasibility. Thirdly, polyhedral constraint trajectory optimization and multi-segment polynomial parameterization (such as minimum time control optimization methods) based on Safe Flight Corridors (SFC) focus primarily on geometric safety and smoothness, lacking a unified and efficient modeling and solution mechanism for imaging-related constraints such as visibility, angular margin, and distance preservation of horizontal / vertical components.

[0004] In terms of cost and constraint handling, mainstream visibility modeling relies on ray casting or the construction of an Euclidean Signed Distance Field (ESDF). The former is susceptible to noise and discrete sampling, making it difficult to obtain stable gradients; the latter requires maintaining a three-dimensional distance field, resulting in high memory and update overhead, which is not conducive to real-time airborne applications. Time allocation often uses fixed ratios or heuristic rules, which are difficult to adaptively adjust with target velocity and environmental complexity, leading to shrinking feasible regions and optimization interruptions. At the same time, the mixed constraints of relative and absolute time of the trajectory are often handled separately, causing inconsistent trade-offs and optimization non-convergence.

[0005] Therefore, there is an urgent need to propose an optimized UAV target tracking method to ensure the real-time performance, security, robustness and engineering deployability of target tracking under complex obstruction and violent maneuvering conditions. Summary of the Invention

[0006] The purpose of this application is to provide a method, apparatus, device, medium, and product for tracking targets using unmanned aerial vehicles (UAVs), which can improve the real-time performance, safety, and robustness of tracking moving targets using UAVs in dynamic environments.

[0007] To achieve the above objectives, this application provides the following solution.

[0008] In a first aspect, this application provides a method for tracking a target using an unmanned aerial vehicle (UAV), comprising: acquiring observation data from airborne sensors; constructing an environmental map based on the airborne sensor observation data, and predicting the target position using a state estimation method based on the airborne sensor observation data to obtain a predicted position sequence of the target; based on spatial visibility reachability constraints, obtaining a set of reachable points corresponding to each predicted position according to the environmental map and the predicted position sequence, and performing segmented path planning using all reachable points corresponding to each predicted position as candidate sub-target points to obtain a relay point corresponding to each predicted position; constructing an initial seed based on the initial position and initial velocity direction of the UAV, and generating a safe flight corridor using an iterative region expansion method based on the initial seed and all relay points; obtaining a trajectory constraint corresponding to each relay point using a perception constraint method based on the environmental map, the predicted position sequence, and all relay points; and obtaining a tracking trajectory using a trajectory optimization method based on the safe flight corridor and the trajectory constraint corresponding to each relay point, thereby enabling the UAV to track the target along the tracking trajectory.

[0009] Secondly, this application provides a UAV target tracking device, comprising: a data acquisition module for acquiring observation data from airborne sensors; a map building and position prediction module for building an environmental map based on the airborne sensor observation data, and predicting the target position using a state estimation method based on the airborne sensor observation data to obtain a predicted position sequence of the target; a path planning module for obtaining a set of reachable points corresponding to each predicted position based on spatial visibility reachability constraints, the environmental map, and the predicted position sequence, and performing segmented path planning using all reachable points corresponding to each predicted position as candidate sub-target points to obtain a relay point corresponding to each predicted position; a flight corridor generation module for constructing an initial seed based on the initial position and initial velocity direction of the UAV, and generating a safe flight corridor using an iterative region expansion method based on the initial seed and all relay points; a trajectory constraint generation module for obtaining a trajectory constraint corresponding to each relay point using a perception constraint method based on the environmental map, the predicted position sequence, and all relay points; and a target tracking module for obtaining a tracking trajectory using a trajectory optimization method based on the safe flight corridor and the trajectory constraints corresponding to each relay point, enabling the UAV to track the target along the tracking trajectory.

[0010] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described UAV target tracking method.

[0011] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described UAV target tracking method.

[0012] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned UAV target tracking method.

[0013] Based on the specific embodiments provided in this application, the following technical effects are disclosed.

[0014] This application provides a method, apparatus, device, medium, and product for UAV target tracking. By predicting the target position, the UAV tracking trajectory can be planned in advance based on the predicted target position, ensuring that the line of sight always leads the target movement, significantly reducing the risk of losing track and improving tracking consistency. Based on the environmental map and the target predicted position sequence, a set of reachable points corresponding to each predicted position is obtained. All reachable points corresponding to each predicted position are used as candidate sub-target points for segmented path planning. This allows all potential sub-target points to be locked in advance within the target movement space, ensuring that segmented path planning always progresses along the feasible region, avoiding repeated replanning caused by real-time obstacle avoidance, and maintaining a continuous and smooth tracking trajectory. An initial UAV tracking path is constructed based on the UAV's initial position and initial velocity direction. The seed is generated using an iterative region expansion method to create a safe flight corridor. This method establishes a convex polyhedron aligned with the dynamic direction during the initial expansion, allowing subsequent safe flight corridors to naturally follow the flight trend, reducing sharp turns and contractions, lowering tracking time, and improving smoothness. A perception constraint method is used to obtain the trajectory constraints corresponding to each relay point. A fan-shaped visible region is constructed in the environment map using the relay points as seeds, and distance-preserving constraints are established for the horizontal and vertical components to ensure that the target remains visible and at a reasonable line-of-sight distance. Based on the safe flight corridor and the trajectory constraints corresponding to each relay point, a trajectory optimization method is used to obtain the tracking trajectory. A multi-constraint unified optimization model is established, which can output a spatiotemporal trajectory that meets the requirements of safety, no occlusion, smoothness, and dynamic feasibility. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is an application environment diagram of a UAV target tracking method according to an embodiment of this application.

[0017] Figure 2 This is a flowchart illustrating a drone target tracking method according to an embodiment of this application.

[0018] Figure 3 This is a schematic diagram of a sector-shaped visible area obtained by using the field-of-view constraint method, provided as an embodiment of this application.

[0019] Figure 4 This is a schematic diagram of the overall framework of a drone target tracking method provided in an embodiment of this application.

[0020] Figure 5 This is a schematic diagram of Gazebo simulation results provided in an embodiment of this application.

[0021] Figure 6 This is a schematic diagram of Rviz simulation results provided for an embodiment of this application.

[0022] Figure 7 This is a schematic diagram of the functional modules of a drone target tracking device provided in another embodiment of this application.

[0023] Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] The UAV target tracking method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 101 communicates with server 102 via a network. A data storage system can store the data that server 102 needs to process. The data storage system can be set up independently, integrated into server 102, or placed in the cloud or on another server. Terminal 101 can send airborne sensor observation data to server 102. After receiving the airborne sensor observation data, server 102 builds an environmental map and predicts the target location. Using all reachable points corresponding to each predicted location as candidate sub-target points, it performs segmented path planning and generates a safe flight corridor. Based on the safe flight corridor and trajectory constraints, it obtains a tracking trajectory and enables the UAV to track the target along the tracking trajectory. Server 102 can feed back the obtained tracking trajectory to terminal 101. Furthermore, in some embodiments, the UAV target tracking method can also be implemented independently by server 102 or terminal 101. For example, terminal 101 can directly process the airborne sensor observation data to obtain the tracking trajectory, or server 102 can obtain the airborne sensor observation data from the data storage system and perform subsequent processing to obtain the tracking trajectory.

[0027] The terminal 101 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 102 can be implemented using a standalone server or a server cluster composed of multiple servers, or it can be a cloud server.

[0028] In one exemplary embodiment, such as Figure 2 As shown, a method for tracking unmanned aerial vehicles (UAVs) targets is provided. This method is executed by a computer device, specifically a terminal or server, or both. In this embodiment, the method is applied to... Figure 1 Taking server 102 as an example, the explanation includes the following steps 201 to 206.

[0029] Step 201: Acquire airborne sensor observation data. In one example, the airborne sensor observation data includes visual positioning data and depth positioning data. In another example, the airborne sensor observation data may also include laser positioning data, or fused data from different positioning sources.

[0030] Step 202: Construct an environmental map based on the airborne sensor observation data, and predict the target position using a state estimation method based on the airborne sensor observation data to obtain the predicted target position sequence. In one example, the state estimation method is the Kalman filter algorithm.

[0031] In one example, the observed position of the target in the current frame is obtained from the airborne sensor, and a discrete linear model with position-velocity as the state is established.

[0032] This example considers the kinematic and dynamic characteristics of an unmanned aerial vehicle (UAV) target tracking system. It assumes the UAV state is described by both position and velocity vectors, and employs a Kalman filter algorithm for recursive estimation and short-time extrapolation. The output of the Kalman filter includes: (i) the optimal estimate of the target's current state (position and velocity); and (ii) a sequence of predicted positions and a set of timestamps for the future finite time domain, used as a reference trajectory for subsequent planning. Observation validity is determined by gating based on the velocity magnitude and covariance threshold obtained from a single "virtual update." When an observation is abnormal (e.g., a sudden increase in velocity exceeding a threshold), the observation is rejected, and the prediction result is maintained, thus avoiding the impact of single-frame noise on the planning process.

[0033] The state transition equations and observation equations of the UAV at discrete moments can be expressed in the following forms.

[0034] .

[0035] .

[0036] in, Here is the state transition matrix. To control the input matrix, For the observation matrix, For the system The state vector at time t, For the system The state vector at time t, For the system Time-based control input, For the system The observation vector at time t, For the system Time-based process noise, For the system Measurement noise at time.

[0037] The prediction (time update) steps of Kalman filtering are as follows.

[0038] .

[0039] .

[0040] in, for The predicted state estimate at time 10:00. for The estimated value of the state at time t. for The covariance matrix of the state estimate at time step. for The covariance matrix of the state estimate at time step. Let be the process noise covariance matrix.

[0041] The measurement update steps are as follows.

[0042] .

[0043] .

[0044] .

[0045] in, for The Kalman gain matrix at time 10:00. To measure the noise covariance matrix, It is the identity matrix. for The updated state estimate at time step. For the updated The covariance matrix of the state estimate at time step.

[0046] Through the above recursive process, the motion state of the UAV can be estimated and predicted in real time under noise interference, providing accurate prior information for trajectory generation and target tracking.

[0047] Step 203: Based on spatial visual reachability constraints, obtain the set of reachable points corresponding to each predicted location according to the environment map and the predicted location sequence, and perform segmented path planning using all reachable points corresponding to each predicted location as candidate sub-target points to obtain the relay point corresponding to each predicted location.

[0048] In one example, the environment map is updated statically or incrementally based on the airborne sensor observation data. Within the environment map, segmented path planning is performed for multiple targets using the predicted position sequence of the UAV target. In this example, [the method is described]. The algorithm performs path planning for each segment. It breaks down the global problem into several smaller-scale iterations. Search. a The next search starts from the endpoint of the previous round and continues with the [number]th [round]. a The set of reachable points corresponding to each predicted location is used as the termination set, thus obtaining the relay point corresponding to each predicted location. This hierarchical strategy significantly reduces the search space and improves online efficiency while ensuring topological rationality.

[0049] Step 204: Construct an initial seed based on the initial position and initial velocity direction of the UAV. Based on the initial seed and all relay points, generate a safe flight corridor using an iterative region expansion method.

[0050] Step 205: Based on the environmental map, the predicted location sequence, and all relay points, a perceptual constraint method is used to obtain the trajectory constraint corresponding to each relay point.

[0051] Step 206: Based on the safe flight corridor and the trajectory constraints corresponding to each relay point, a trajectory optimization method is used to obtain the tracking trajectory, enabling the UAV to track the target along the tracking trajectory. Specifically, the trajectory optimization method is the Minimum Control Optimization (MINCO) method.

[0052] In one example, step 203 above can be replaced by steps 2031 to 2033.

[0053] Step 2031: Based on the environmental map and the predicted location sequence, a visibility calculation method is used to obtain the set of visible points corresponding to each predicted location.

[0054] Step 2032: Perform distance filtering on the set of visible points corresponding to each predicted location to obtain the set of reachable points corresponding to each predicted location.

[0055] Step 2033: Perform segmented path planning using all reachable points corresponding to each predicted location as candidate sub-target points to obtain the relay point corresponding to each predicted location.

[0056] Specifically, obtaining the predicted location sequence of the drone target. And the set of visible points corresponding to each predicted location. To predict the position sequence of the first k One predicted location, This represents the total number of predicted locations. Considering both distance and occlusion, for each... Define a set of reachable points As the first Candidate sub-target points for sub-segment path planning.

[0057] A greedy layered strategy is adopted to break down the global problem into smaller parts. The first sequential small-scale search: k Second-rate The algorithm takes the end point of the previous round as the starting point and... For candidate sub-target points, obtain the relay point corresponding to each predicted location. . No. k Second-rate In the algorithm, the cost function and heuristic functions The definition is shown in the following formula.

[0058] .

[0059] .

[0060] in, For the first k Second-rate The algorithm runs from the starting node to the current node. n The actual cost, For the first k Second-rate Nodes in the algorithm n exist x and y Direction to predicted position distance, For the first k Second-rate Nodes in the algorithm n exist z Direction to predicted position distance, This represents the desired viewing distance.

[0061] In each path planning segment, nodes that collide are directly judged as infeasible. k The search terminates when the current node is reached. The relay points obtained in each search As the starting point for the next search, until a thorough understanding is achieved. The global path.

[0062] In this example, The algorithm's heuristic function comprehensively considers the horizontal and vertical distance deviation from the target node, as well as reachability. Nodes located inside obstacles or on collision boundaries are directly deemed infeasible and pruned.

[0063] In one example, step 204 above can be replaced by steps 2041 to 2044.

[0064] Step 2041: Connect all relay points sequentially to obtain the guiding path.

[0065] Step 2042: Based on the initial seed, perform convex expansion using the feasible region convex expansion method to obtain the convex polyhedron corresponding to the first iteration.

[0066] Step 2043: For the p-th iteration, obtain the seed for the p-th iteration based on the end of the convex polyhedron corresponding to the (p-1)-th iteration and the exit point of the convex polyhedron corresponding to the (p-1)-th iteration with the guide path. Where p > 1. The exit point is the first intersection point where the guide path leaves the boundary of the convex polyhedron.

[0067] Step 2044: Based on the seed of the p-th iteration, perform convex expansion using the feasible region convex expansion method to obtain the convex polyhedron corresponding to the p-th iteration; if the convex polyhedron corresponding to the p-th iteration covers the last relay point, then connect all the convex polyhedra in sequence to generate a safe flight corridor; otherwise, perform the p+1-th iteration.

[0068] In this example, multiple convex polyhedra are adaptively generated and concatenated along the guidance path to form a SafeFlight Corridor (SFC), serving as a geometrically feasible region to ensure geometric feasibility and obstacle avoidance capabilities. The first convex polyhedron is "pulled out" from a short line segment along the initial velocity direction of the UAV to conform to the current motion trend. Subsequent convex polyhedra are generated by rolling around the intersection of the "end of the previous convex polyhedron - the guidance path and the current convex polyhedron" as the base segment. To avoid excessive overlap and gap leakage, the scale of the convex polyhedrons is automatically reduced and the number of segments is increased when the local curvature increases or the environment narrows, ensuring connectivity and coverage. The SFC is represented using linear inequalities for easy direct use in subsequent optimizations.

[0069] Specifically, based on the initial position and initial velocity direction of the UAV, a short line segment is generated from the starting point along the velocity direction as the first guiding line, which serves as the initial seed. The feasible region convex dilatation method is then used on both sides of the initial seed to generate the convex polyhedron corresponding to the first iteration, determining its normal and scale parameters.

[0070] The next convex polyhedron is generated based on the line segment formed by the intersection of the end of the previous convex polyhedron, the guiding path, and the previous convex polyhedron. When the local curvature of the path is large or the environment is restrictive, the polyhedron scale is automatically reduced and the number of segments is increased to avoid excessive overlap and leakage. The final result is a safe flight corridor as shown in the following equation. .

[0071] .

[0072] in, For safe flight corridors, The number of convex polyhedra. For the first A convex polyhedron It is a spatial position vector. For the first A plane normal vector matrix, For the first The constant term of a plane equation.

[0073] During the construction of the safe flight corridor, the intersection of each convex polyhedron with the environment occupies is detected. If a certain segment is not feasible, the search is reversed to the nearest feasible segment and the local search is repeated.

[0074] In one example, the trajectory constraints include view constraints, distance constraints, and visibility constraints. Step 205 specifically includes the following steps: for any relay point, based on the environment map, the relay point, and the predicted location corresponding to the relay point, the view constraint corresponding to the relay point is obtained using a view constraint method, the distance constraint corresponding to the relay point is obtained using a distance constraint method, and the visibility constraint corresponding to the relay point is obtained using a visibility constraint.

[0075] In this example, for any relay point, a fan-shaped visible area aligned with the line-of-sight is constructed based on that relay point. An angular margin is set to limit the angle between the UAV, target, and line of sight, reducing the risk of occlusion and excessive shooting angle. Simultaneously, the distance between the UAV and the target is decomposed into horizontal and vertical components, each constrained separately. The horizontal component is limited to a desired range, and a smooth penalty function is used to impose a cost on deviations outside the range. The vertical component has an upper bound tolerance to balance platform pitch limitations and image quality. This results in two parts: a distance preservation cost and an unobstructed area cost, which can be linearly superimposed with weights to form a comprehensive visibility-distance cost.

[0076] Specifically, for each predicted location and its corresponding relay points Calculate the angle bisector vector This generates a sector-shaped visible area as shown in the following formula.

[0077] .

[0078] in, The visible area is a fan shape. This is the threshold for the angle of view.

[0079] The fan-shaped visible area generated in this example is as follows: Figure 3 As shown in the figure. Here, A and B are obstacles, W is a relay point, H is the predicted position sequence of the target, and region Q is the fan-shaped visible area obtained using the view constraint method.

[0080] Set up relay points separately The horizontal and vertical distance components to the target are and The expected horizontal distance range is The upper limit of vertical tolerance is For the horizontal component, use Smooth penalty function Its implementation is as follows.

[0081] .

[0082] in, This represents the lower limit of the expected distance range. This represents the upper limit of the desired distance range. It is a small constant. Based on the above equation, the cost function of the distance constraint is... As shown in the following formula.

[0083] .

[0084] Given an angular margin, the visibility constraint is equivalent to the unobstructed criterion shown in the following equation. .

[0085] .

[0086] in, This is the angle margin parameter.

[0087] Further sampling penalty is applied to obtain the visibility cost function as shown in the following equation. .

[0088] .

[0089] in, To optimize the obtained UAV 3D trajectory and The corresponding position.

[0090] set up ,but and The gradients can all be in the complexity Internal calculation.

[0091] .

[0092] in, To optimize the obtained UAV 3D trajectory i The polynomial coefficient vector of the segment is used to control the shape of the trajectory. To optimize the obtained UAV 3D trajectory i The time length variable of the segment is used to control the time allocation.

[0093] In one example, the MINCO multi-segment polynomial is used to compactly parameterize the 3D trajectory. The entire trajectory is uniquely determined by all relay points and the duration of each segment, and the coefficients are calculated by linear operators, exhibiting linear complexity and second-order continuity. Under constraints such as SFC, visibility-distance, and dynamic feasibility (upper bound of velocity / acceleration), a unified optimization model is established with the objective of minimizing the trajectory third-order derivative (jerk) energy and the weighted sum of total time. To eliminate the inequality constraint that "total duration is not less than predicted duration," time reparameterization is introduced: free variable mapping is used to satisfy the constraints, thereby searching for the optimal solution in an inequality-free parameter space, improving feasibility and convergence stability. The gradient value is obtained by utilizing the analytical gradient interface of MINCO and combining it with the chain rule; a first-order or quasi-Newton method is used in conjunction with line search for joint iteration. Two strategies are adopted for constraints during the optimization process: first, explicit geometric constraints (trajectory sampling points are located inside each polyhedron); second, relative / absolute time penalties are used to handle dynamic feasibility and visibility / distance terms respectively, taking into account both intra-segment integral constraints and discrete sampling constraints. The iteration terminates when the objective function converges or the maximum number of iterations reaches a preset value, and outputs an executable spatiotemporal trajectory and segment duration allocation.

[0094] Specifically, the total duration inequality constraint is eliminated by time reparameterization, and a multi-constraint unified optimization model is established with the objective of minimizing the weighted sum of the trajectory jerk and the total time. The output is a spatiotemporal trajectory that satisfies safety, occlusion-free characteristics, smoothness, and dynamic feasibility. The specific implementation steps are as follows.

[0095] In this example, the MINCO trajectory class The definition is as follows.

[0096] .

[0097] The trajectory is represented in segments as follows.

[0098] .

[0099] in, To optimize the obtained 3D trajectory of the UAV, m for Dimensions T To optimize the trajectory termination time, For the MINCO function, q For relay point vectors, M The number of segments, c for The trajectory coefficient matrix, To optimize the obtained three-dimensional trajectory of the UAV i Segmentation, c i for The trajectory coefficient matrix, For the basis function vector, To optimize the obtained three-dimensional trajectory of the UAV i The duration of the segment.

[0100] Optimized cost function as follows.

[0101] .

[0102] in, As a time penalty weight, This is the median of the total duration. and These represent the weights of the distance constraint and the visibility constraint, respectively. In the above equation, the first term on the right-hand side is the energy of the trajectory jerk, and the second term on the right-hand side is the total time penalty.

[0103] The time reparameterization formula is as follows.

[0104] .

[0105] .

[0106] .

[0107] in, For the target prediction duration, Reparameterize the variable for time. To optimize the obtained three-dimensional trajectory of the UAV i Segmented prediction duration, for The corresponding time parameterized variable, To optimize the obtained three-dimensional trajectory of the UAVM Segmented prediction duration.

[0108] By optimizing the cost function and combining it with the following gradient relationship, the optimal tracking trajectory of the UAV can be obtained.

[0109] .

[0110] .

[0111] .

[0112] in, for The corresponding time is then parameterized as a variable.

[0113] In one example, as follows Figure 4 The method shown is used for UAV target tracking. First, airborne sensor data is acquired, and the predicted position sequence of the target is determined based on the airborne sensor data. Then, a layered approach is adopted. The algorithm performs path search and constructs a safe flight corridor. It then optimizes the trajectory by combining the safe flight corridor with trajectory constraints, and tracks the target according to the optimized trajectory.

[0114] In this example, Gazebo software is used to build such as Figure 5 Experiments were conducted to verify the complex scenario shown, which included dense cylindrical obstacles and narrow passages. Figure 5 The demonstration showcases a typical complex tracking scenario where the target is obstructed, maneuverable, and in a confined environment. Under these circumstances, the drone needs to maintain continuous visibility of the target and ensure flight safety.

[0115] In Gazebo software, targets were set up for sharp turns and variable-speed maneuvers. In Rviz tools, the occupied grid, guide path, scrolling safe flight corridor, visibility sector, and optimized spatiotemporal trajectory were simultaneously visualized. The planner ran a pipeline of "prediction-hierarchical guided search-scrolling SFC-MINCO optimization" online at fixed intervals. Local re-search and corridor refresh were triggered when occlusion or sudden environmental changes occurred; if observations were missing, prediction was maintained, and optimization continued seamlessly after recovery. During the experiment, sensor noise settings and control limits were uniform. Key metrics recorded included: mission success / collision rate, target visibility (percentage of unoccluded frames and minimum line-of-sight angle), horizontal / vertical distance error, trajectory smoothness (jerk energy), average planning delay, and number of replanning triggers. Rviz simulation results are as follows: Figure 6 As shown, by obtaining the predicted position sequence of the target, and considering both visual accessibility and obstacle constraints, a result was obtained that... Figure 5 The optimized tracking trajectory planning results corresponding to the simulation scenario.

[0116] Based on the same inventive concept, this application also provides a drone target tracking device for implementing the drone target tracking method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more drone target tracking device embodiments provided below can be found in the limitations of the drone target tracking method described above, and will not be repeated here.

[0117] In one exemplary embodiment, such as Figure 7 As shown, a drone target tracking device is provided, including: a data acquisition module 701, a map building and location prediction module 702, a path planning module 703, a flight corridor generation module 704, a trajectory constraint generation module 705, and a target tracking module 706.

[0118] The system includes the following modules: a data acquisition module 701, which acquires observation data from airborne sensors; a map building and location prediction module 702, which builds an environmental map based on the airborne sensor observation data and uses a state estimation method to predict the target's location, resulting in a predicted location sequence; a path planning module 703, which, based on spatial visibility reachability constraints, obtains a set of reachable points corresponding to each predicted location according to the environmental map and the predicted location sequence, and uses all reachable points corresponding to each predicted location as candidate sub-target points for segmented path planning, resulting in a relay point corresponding to each predicted location; a flight corridor generation module 704, which constructs an initial seed based on the UAV's initial position and initial velocity direction, and uses an iterative region expansion method based on the initial seed and all relay points to generate a safe flight corridor; and a trajectory constraint generation module 705, which uses a perception constraint method based on the environmental map, the predicted location sequence, and all relay points to obtain the trajectory constraint corresponding to each relay point. The target tracking module 706 is used to obtain a tracking trajectory based on the safe flight corridor and the trajectory constraints corresponding to each relay point, using a trajectory optimization method, so that the UAV tracks the target along the tracking trajectory.

[0119] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 8As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores observation data from airborne sensors. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network. When executed by the processor, the computer program implements a UAV target tracking method.

[0120] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0121] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0122] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0123] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0124] The beneficial effects of this application are as follows.

[0125] To address the problems of occlusion, distance mismatch, and dynamic infeasibility in existing UAV target tracking methods in complex scenarios, this application proposes a UAV target tracking method based on multi-constraint optimization, which can stably output executable trajectories under conditions of severe target maneuvering and complex environmental occlusion.

[0126] The planner executes each step of this application in a fixed-cycle manner. When a target is detected to be rapidly approaching, or when sudden occlusion or environmental changes cause SFC failure, a local re-search and corridor refresh are triggered. When the number of missing observations exceeds a threshold, the planner switches to prediction hold + recapture mode. Once observations are recovered, optimization continues on the current state to ensure that the task is not interrupted.

[0127] This application proposes a "prediction-driven hierarchical guided search-rolling safety corridor" mechanism in the path search and geometric feasible region construction stages: taking the endpoint of the previous round as the starting point, the set of unobstructed / visible objects corresponding to the k-th target pose obtained by Kalman prediction is set as the termination set, thus decomposing the global problem into several smaller-scale steps. The search retrieves relay points to guide the path. A heuristic function, combined with the horizontal / vertical distance deviation from the target and reachability, directly prunes nodes located inside obstacles or at collision boundaries. Safe Flight Corridors (SFCs) are formed by adaptively rolling and connecting convex polyhedra along this guiding path. The first polyhedron is "pulled out" along the current velocity direction, and subsequent polyhedra are updated using the "end of the upper segment - intersection of the path and the polyhedron" as the base segment. When local curvature increases or the channel narrows, the scale is automatically reduced and segments are added. SFCs are represented using linear inequalities for easy optimization and direct invocation. This mechanism significantly compresses the search space and improves online efficiency while ensuring topological rationality, and enhances the robustness and executability of the geometrically feasible region by suppressing excessive overlap and gap leakage.

[0128] Furthermore, this application proposes a differentiable dual constraint of "visibility-distance" in the constraint modeling stage: constructing a fan-shaped unobstructed region aligned with the line-of-sight direction using visibility keypoints as seeds, and setting a line-of-sight angle margin to limit the "UAV-target-line-of-sight" angle. Simultaneously, the UAV-target distance is decomposed into horizontal and vertical components with separate constraints. The horizontal component is confined outside the desired range to suppress deviation through a smooth penalty, while the vertical component has an upper bound tolerance to balance attitude constraints and imaging quality. These two constraints are linearly superimposed with weights to form a unified visibility-distance comprehensive cost, seamlessly embedded in MINCO trajectory optimization and coordinated with dynamic constraints such as velocity / acceleration upper bounds. This design avoids the viewpoint degradation and occlusion loss of control caused by a single Euclidean distance, significantly improving tracking stability, imaging quality, and solution convergence in complex occlusion and target maneuvering scenarios.

[0129] Through the aforementioned steps of fusion target prediction, hierarchical guided search, rolling safe flight corridor generation, and MINCO optimization, this application realizes a real-time, safe, and visibility-maintaining target tracking method for UAVs facing moving targets in dynamic environments.

[0130] This application is applicable to UAV target tracking tasks in complex environments. It comprehensively considers various constraints such as obstacle avoidance, visibility assurance, target distance maintenance, dynamic feasibility, and trajectory smoothness, achieving stable and reliable target tracking. This application obtains the future position sequence of the target through target detection and motion prediction, constructs a safe flight corridor by combining path search based on perceived occlusion, and establishes a visibility area and distance maintenance model, forming a multi-constraint optimization problem. Utilizing a multi-segment polynomial trajectory parameterization method and time constraint elimination technology, the constraints of dynamic feasibility, obstacle avoidance, occlusion-free operation, and distance maintenance are solved under a unified optimization framework, generating a tracking trajectory that satisfies spatiotemporal optimality. This application can improve trajectory smoothness and real-time performance while ensuring safety and visibility, reducing the risk of occlusion and collision. It is suitable for application scenarios requiring continuous target tracking by UAVs, such as security patrol, emergency rescue, and film shooting.

[0131] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0132] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0133] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0134] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0135] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for tracking targets using an unmanned aerial vehicle (UAV), characterized in that, The UAV target tracking method includes: Acquire observation data from airborne sensors; An environmental map is constructed based on the airborne sensor observation data, and the target position is predicted using a state estimation method based on the airborne sensor observation data, resulting in a predicted target position sequence. Based on spatial visual reachability constraints, the set of reachable points corresponding to each predicted location is obtained according to the environmental map and the predicted location sequence. Then, segmented path planning is performed using all reachable points corresponding to each predicted location as candidate sub-target points to obtain the relay point corresponding to each predicted location. An initial seed is constructed based on the initial position and initial velocity direction of the UAV. Based on the initial seed and all relay points, an iterative region expansion method is used to generate a safe flight corridor. Based on the environmental map, the predicted location sequence, and all relay points, a perceptual constraint method is used to obtain the trajectory constraint corresponding to each relay point; Based on the safe flight corridor and the trajectory constraints corresponding to each relay point, a trajectory optimization method is used to obtain the tracking trajectory, enabling the UAV to track the target along the tracking trajectory.

2. The UAV target tracking method according to claim 1, characterized in that, The state estimation method is the Kalman filter algorithm.

3. The UAV target tracking method according to claim 1, characterized in that, Based on spatial visual reachability constraints, a set of reachable points corresponding to each predicted location is obtained according to the environmental map and the predicted location sequence. Then, segmented path planning is performed using all reachable points corresponding to each predicted location as candidate sub-target points to obtain the relay point corresponding to each predicted location, specifically including: Based on the environmental map and the predicted location sequence, a visibility calculation method is used to obtain the set of visible points corresponding to each predicted location; Distance filtering is performed on the set of visible points corresponding to each predicted location to obtain the set of reachable points corresponding to each predicted location; Segmented path planning is performed using all reachable points corresponding to each predicted location as candidate sub-target points to obtain the relay point corresponding to each predicted location.

4. The UAV target tracking method according to claim 1, characterized in that, An initial seed is constructed based on the initial position and initial velocity direction of the UAV. Then, based on the initial seed and all relay points, an iterative region expansion method is used to generate a safe flight corridor, specifically including: Connect all the relay points in sequence to obtain the guiding path; Based on the initial seed, convex expansion is performed using the feasible region convex expansion method to obtain the convex polyhedron corresponding to the first iteration. For the p-th iteration, the seed for the p-th iteration is obtained based on the end of the convex polyhedron corresponding to the (p-1)-th iteration and the intersection of the convex polyhedron corresponding to the (p-1)-th iteration and the exit point of the guiding path; p>1. Based on the seed of the p-th iteration, convex expansion is performed using the feasible region convex expansion method to obtain the convex polyhedron corresponding to the p-th iteration. If the convex polyhedron corresponding to the p-th iteration covers the last relay point, then all convex polyhedra are sequentially connected to generate a safe flight corridor; otherwise, the p+1-th iteration is performed.

5. The UAV target tracking method according to claim 1, characterized in that, The trajectory constraints include viewpoint constraints, distance constraints, and visibility constraints; Based on the environmental map, the predicted location sequence, and all relay points, a perceptual constraint method is used to obtain the trajectory constraints corresponding to each relay point, specifically including: For any relay point, based on the environment map, the relay point, and the predicted location corresponding to the relay point, the view constraint corresponding to the relay point is obtained by using the view constraint method, the distance constraint corresponding to the relay point is obtained by using the distance constraint method, and the visibility constraint corresponding to the relay point is obtained by using the visibility constraint.

6. The UAV target tracking method according to claim 1, characterized in that, The trajectory optimization method is a minimum control optimization method.

7. A drone target tracking device, characterized in that, The UAV target tracking device includes: The data acquisition module is used to acquire observation data from airborne sensors; The map building and location prediction module is used to build an environmental map based on the airborne sensor observation data, and to predict the target location using a state estimation method based on the airborne sensor observation data, thereby obtaining the predicted location sequence of the target. The path planning module is used to obtain the set of reachable points corresponding to each predicted location based on the spatial visual reachability constraints, according to the environment map and the predicted location sequence, and to perform segmented path planning using all reachable points corresponding to each predicted location as candidate sub-target points to obtain the relay point corresponding to each predicted location. The flight corridor generation module is used to construct an initial seed based on the initial position and initial velocity direction of the UAV, and to generate a safe flight corridor based on the initial seed and all relay points using an iterative region expansion method. The trajectory constraint generation module is used to obtain the trajectory constraint corresponding to each relay point based on the environment map, the predicted location sequence and all relay points, using a perceptual constraint method. The target tracking module is used to obtain a tracking trajectory based on the safe flight corridor and the trajectory constraints corresponding to each relay point, using a trajectory optimization method, so that the UAV tracks the target along the tracking trajectory.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the UAV target tracking method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the UAV target tracking method according to any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the UAV target tracking method according to any one of claims 1-6.

Images