Unmanned aerial vehicle target tracking and intelligent route planning method and system based on YOLO and DSM

Abstract

The application discloses a UAV target tracking and intelligent route planning method based on YOLO and DSM, which comprises the following steps: a UAV collects video streams, thermal images and distance information in real time through multi-modal sensors, fuses multi-modal data by using Kalman filtering, detects targets by using a YOLOv11 model, and outputs position, category and confidence information of the targets; a DeepSORT algorithm is used to continuously track the targets, the target motion trajectory is predicted in combination with Kalman filtering, and the matching between target detection and tracking trajectory is realized by using a Hungarian algorithm, so that stable tracking in a complex environment is ensured; terrain elevation information provided by DSM is used, an improved A* algorithm is combined to plan a flight path of the UAV, so that the UAV can avoid obstacles and continuously track the target in a complex terrain environment; and through a reinforcement learning algorithm, the UAV can dynamically adjust the route planning according to the motion state of the target and the environmental changes, so that when the target is lost or the environmental obstacles are newly added, the system can automatically re-plan the route.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to a method for UAV target tracking and intelligent flight path planning based on the YOLO11 target detection algorithm and digital surface model (DSM). Background Technology

[0002] With the rapid development of drone technology, drones are increasingly being used in target tracking, environmental monitoring, disaster relief, and other fields. However, target tracking and flight path planning in complex environments still face many challenges:

[0003] 1. In dynamic environments, the appearance, posture, and trajectory of a target may change drastically, and traditional target detection and tracking algorithms struggle to cope with problems such as target loss and occlusion in complex scenarios;

[0004] 2. Drones are usually equipped with multiple sensors (such as cameras, infrared sensors, radar, etc.), and how to efficiently fuse multimodal data to improve the robustness of target detection and tracking remains a technical challenge;

[0005] 3. Flight path planning in complex terrain environments: In complex terrain environments, UAVs need to avoid obstacles (such as buildings, trees, etc.) and maintain continuous tracking of the target. Traditional flight path planning algorithms are difficult to balance the multiple requirements of terrain constraints, dynamic obstacle avoidance and target tracking.

[0006] 4. During flight, drones may encounter unexpected situations such as target loss or the addition of environmental obstacles. Traditional flight path planning methods lack sufficient adaptive capabilities and are difficult to adjust the flight path in real time.

[0007] To address the aforementioned issues, some solutions already exist in existing technologies, such as deep learning-based object detection algorithms (e.g., the YOLO series), Kalman filter-based object tracking algorithms, and A* algorithm-based route planning methods. However, these methods still have certain limitations in object tracking and route planning in complex environments, especially in multimodal data fusion, dynamic obstacle avoidance, and adaptive route adjustment. Summary of the Invention

[0008] To overcome the shortcomings of the existing technologies, this invention provides a UAV target tracking and intelligent route planning method based on YOLO and DSM. By combining the YOLOv11 target detection algorithm, DeepSORT target tracking algorithm, high-precision digital surface model (DSM), and improved A* algorithm, it can effectively cope with target tracking and route planning problems in complex environments and significantly improve the autonomy and adaptability of UAVs.

[0009] According to one aspect of the present invention, a method for unmanned aerial vehicle (UAV) target tracking and intelligent route planning based on YOLO and DSM is provided, comprising:

[0010] The system acquires video streams, thermal imaging, and distance information in real time using multimodal sensors, fuses the multimodal data using Kalman filtering, and detects targets using the YOLOv11 model, outputting the target's location, category, and confidence information.

[0011] The DeepSORT algorithm is used to continuously track the target, combined with Kalman filtering to predict the target's trajectory, and the Hungarian algorithm is used to match the target detection and tracking trajectory.

[0012] The flight path of the UAV is planned by using the terrain elevation information provided by DSM and combining it with the improved A* algorithm.

[0013] Based on reinforcement learning algorithms, the flight path planning is dynamically adjusted according to the target's motion state and environmental changes.

[0014] As a further technical solution, the YOLOv11 model introduces an attention mechanism and lightweight design to optimize the network structure, while Focal Loss is used to optimize the loss function;

[0015] An adaptive model update mechanism is introduced to dynamically adjust model weights based on changes in the target's appearance.

[0016] A target loss prediction mechanism is introduced to estimate the probability of target loss by analyzing target movement trends and environmental changes.

[0017] As a further technical solution, an adaptive model update mechanism is introduced.

[0018]

[0019] in, Indicates the current model weights. For learning rate, This is the gradient of the loss function.

[0020] As a further technical solution, the model weights are updated in real time by dynamically adjusting the learning rate. for:

[0021]

[0022] in, The initial learning rate, For sensitivity parameters to changes in appearance, For stability decay parameters, For the duration of stable target tracking, For comprehensive appearance change indicators.

[0023] As a further technical solution, a target loss prediction mechanism is introduced.

[0024]

[0025] in, For the prediction function, For the target movement trend, Due to environmental change, This represents the probability of target loss.

[0026] As a further technical solution, the prediction function is implemented using a neural network model trained on historical data, and the neural network model adopts a three-layer fully connected neural network structure.

[0027] As a further technical solution, the target motion trend includes: motion speed change trend, motion direction change trend, motion trajectory prediction error, and target bounding box change trend;

[0028] The environmental changes include: changes in lighting, changes in background complexity, detection of obstructions, and changes in weather conditions.

[0029] As a further technical solution, the flight path of the UAV is planned by utilizing the terrain elevation information provided by DSM and combining it with an improved A* algorithm, including:

[0030] Let the number of nodes in the drone flight path be... The cost function of the A* algorithm Defined as:

[0031]

[0032] in, From the starting point to the node The actual cost, From node Heuristic cost estimation to the target point;

[0033] Among them, the cost letter and heuristic functions Improvements were made by combining terrain elevation information and dynamic obstacle information:

[0034]

[0035]

[0036] in, It is a node To the node The planar distance; It is a node To the node The change in altitude; It is a node The cost of obstacles; , , It is a weighting parameter used to balance distance, height variation, and obstacle cost; These are the coordinates of the target point; It is a node The coordinates.

[0037] According to one aspect of the present invention, a UAV target tracking and intelligent flight path planning system based on YOLO and DSM is provided, comprising:

[0038] The target detection module is used to acquire video streams, thermal imaging and distance information in real time through multimodal sensors, fuse multimodal data using Kalman filtering, detect targets using the YOLOv11 model, and output the target's position, category and confidence information;

[0039] The target tracking module is used to continuously track the target using the DeepSORT algorithm, combined with Kalman filtering to predict the target's trajectory, and uses the Hungarian algorithm to match the target detection with the tracking trajectory.

[0040] The path planning module is used to plan the flight path of the UAV using the terrain elevation information provided by DSM and the improved A* algorithm.

[0041] The dynamic adjustment module is used to dynamically adjust the flight path planning based on the target's motion state and environmental changes, using reinforcement learning algorithms.

[0042] According to one aspect of the present invention, a non-transitory computer-readable storage medium is provided, the non-transitory computer-readable storage medium storing computer instructions that cause the computer to execute the UAV target tracking and intelligent route planning method based on YOLO and DSM.

[0043] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0044] 1. This invention proposes a novel method for UAV target tracking and intelligent flight path planning by combining the YOLOv11 target detection algorithm, the DeepSORT target tracking algorithm, the high-precision digital surface model (DSM), and the improved A* algorithm. This method can effectively address the target tracking and flight path planning problems in complex environments and significantly improve the autonomy and adaptability of UAVs.

[0045] 2. The method of this invention combines deep learning target detection algorithms with high-precision geospatial data to achieve accurate target tracking and autonomous navigation of UAVs in complex environments. It can significantly improve the autonomy and adaptability of UAVs in target tracking tasks and has broad application prospects. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart illustrating the UAV target tracking and intelligent route planning method based on YOLO and DSM provided in an embodiment of the present invention. Detailed Implementation

[0048] Unless otherwise defined, the technical or scientific terms used herein shall have the ordinary meaning as understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” “third,” and similar terms used in this patent application specification and claims do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an” or “a” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “comprising” or “including” and similar terms mean that the elements or objects preceding “comprising” or “including” encompass the elements or objects listed following “comprising” or “including” and their equivalents, and do not exclude other elements or objects. The terms “connected” or “linked” and similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined to form new technical solutions. Such combinations are not bound by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0050] This invention provides a method for UAV target tracking and intelligent flight path planning based on YOLO11 and digital surface models. Its core lies in combining deep learning target detection algorithms with high-precision geospatial data to achieve accurate target tracking and autonomous navigation of the UAV in complex environments. (Refer to...) Figure 1 The specific implementation method is described in detail below.

[0051] Step S1: Target detection based on YOLO11 fused data.

[0052] First, the drone uses its onboard high-definition cameras, infrared sensors, radar, and other multimodal sensors to collect video stream data, thermal imaging data, and distance information in real time, and then fuses this data before transmitting it to the onboard computing unit.

[0053]

[0054] in, This refers to video stream data captured by a high-definition camera. This represents thermal imaging data collected by an infrared sensor. This represents the distance information acquired by the radar, which includes various information such as the observed values ​​and observation noise, as described below.

[0055] The system model is as follows:

[0056] The Kalman filter fusion algorithm is used to fuse multimodal data into a unified input. Assume the system has... There are 10 sensors, each providing different observation data. Let's assume at time 10... The system's state vector is The state transition matrix (which describes how the system state changes over time) is: The process noise is (Usually assumed to be zero-mean Gaussian noise, the corresponding covariance matrix is...) Then the state equation of the system can be expressed as:

[0057]

[0058] The observation model is:

[0059] At any moment For different sensors (such as cameras, infrared sensors, radar, etc.), the first indivual The sensor's observations are denoted as , will the indivual The observation matrix corresponding to the sensor (describing how the observations are extracted from the state vector) is denoted as... , will the indivual The sensor's observation noise (usually assumed to be zero-mean Gaussian noise, with a corresponding covariance matrix of...) ) is denoted as Then the observation equation for each sensor is:

[0060]

[0061] in, It is the first The observation noise of each sensor, with a covariance matrix of... ;

[0062] Kalman filtering-based multimodal data fusion is as follows:

[0063] During the prediction phase, the system estimates the state based on the previous time step. Covariance Matrix Predict the state at the current moment. Covariance Matrix :

[0064]

[0065]

[0066] During the update phase, the system updates based on the observations from each sensor. The state estimate and covariance matrix are updated progressively. For the ... One sensor, Kalman gain The calculation formula is:

[0067]

[0068] Then, using the first Observations from each sensor Update state estimation Covariance Matrix :

[0069]

[0070]

[0071] in, It is an identity matrix.

[0072] For multiple sensors, the updates can be performed sequentially using the observations from each sensor. Let there be... The update process for a sensor can be represented as follows:

[0073]

[0074]

[0075] for arrive The following updates will be performed sequentially:

[0076]

[0077]

[0078]

[0079] Finally, after updating all the sensors, the fused state estimate is obtained. Covariance Matrix :

[0080]

[0081]

[0082] Through the steps described above, Kalman filtering can fuse observation data from multiple sensors to obtain a more accurate state estimate. The core idea of ​​multimodal data fusion is to utilize the complementary information from different sensors and, through the prediction and update mechanism of Kalman filtering, gradually correct the state estimate, thereby improving the robustness and accuracy of the system.

[0083] Video frames, which have undergone multimodal fusion processing, are input into a YOLO11-based object detection network (here, "video frames" refers to the data after multimodal fusion processing). The network, pre-trained on a large-scale, diverse dataset, efficiently identifies target objects in video frames from fused data and outputs the target's location, category, and confidence information. The YOLO11 model significantly improves the accuracy and speed of target detection by optimizing the network structure (e.g., introducing attention mechanisms and lightweight design) and loss function (e.g., using Focal Loss to address class imbalance), making it particularly suitable for real-time detection needs of UAVs in dynamic environments. Furthermore, the system introduces an adaptive model update mechanism that dynamically adjusts model weights based on changes in target appearance (e.g., lighting conditions, target pose), further enhancing detection robustness.

[0084] The output of the YOLO11 model can be represented as:

[0085]

[0086] in, Indicates the center coordinates of the target bounding box. This indicates the width and height of the bounding box. Indicates the target category. Indicates the confidence level of the target.

[0087] YOLO11 significantly improves detection efficiency by introducing attention mechanisms (such as the SE module) and lightweight design (such as depthwise separable convolution). Its loss function, Focal Loss, is defined as follows:

[0088]

[0089] in, This represents the target probability predicted by the model. and It is a hyperparameter used to solve the class imbalance problem.

[0090] The adaptive model update mechanism dynamically adjusts the model weights. To adapt to changes in the target's appearance. The update formula is:

[0091]

[0092] in, Indicates the current model weights. For learning rate, This represents the gradient of the loss function. The learning rate is dynamically adjusted. By updating the model weights in real time, the system can maintain high detection accuracy when the target's appearance changes (such as changes in lighting or target pose).

[0093] The learning rate is dynamically adjusted when the target's appearance changes (such as changes in lighting or target pose). The specific methods are as follows:

[0094] The method for detecting changes in illumination is as follows:

[0095]

[0096] in, Indicates the pixel intensity of the target region in the current frame. Indicates the pixel intensity of the target region in the reference frame. The function calculates the average brightness of the image.

[0097] The method for detecting attitude changes is as follows:

[0098]

[0099] in, and These represent the target bounding box parameters for the current frame and the reference frame, respectively.

[0100] The method for detecting changes in feature similarity is as follows:

[0101]

[0102] in, and These represent the depth feature vectors extracted from the current frame and the reference frame, respectively.

[0103] The overall appearance change indicators are as follows:

[0104]

[0105] Where w1, w2, and w3 are weight parameters, satisfying .

[0106] The system dynamically adjusts the learning rate based on the degree of change in the overall appearance.

[0107]

[0108] in, The initial learning rate, For sensitivity parameters to changes in appearance, For stability decay parameters, The duration of stable tracking of the target.

[0109] To further enhance the robustness of target detection and tracking, a target loss prediction mechanism is introduced into the system. This is achieved by analyzing the target's movement trend. and environmental changes To estimate the probability of target loss :

[0110]

[0111] in, This is the prediction function used to calculate the probability of target loss. When... When the threshold is exceeded, the system adjusts the drone's attitude or flight path in advance to reduce the probability of target loss.

[0112] The target motion trend T is quantitatively analyzed through the following multiple dimensions:

[0113] Trend of change in motion speed:

[0114]

[0115] in, Indicates the target at time. The speed of movement is calculated by the change in target position between consecutive frames.

[0116] Trend of change in direction of movement:

[0117]

[0118] in, Indicates the target at time. The direction and angle of movement.

[0119] Motion trajectory prediction error:

[0120]

[0121] Where pos_predicted(t) is the target position predicted by the Kalman filter, and pos_actual(t) is the actual detected target position.

[0122] Trend of target bounding box change:

[0123]

[0124] in, This represents the area of ​​the target bounding box.

[0125] The comprehensive sports trend index is:

[0126]

[0127] in, , , , For the weight parameters, satisfying + + + = 1.

[0128] Environmental change (E) is quantified using the following factors:

[0129] Changes in light:

[0130]

[0131] in, This indicates the brightness distribution of the image in the current frame.

[0132] Background complexity changes:

[0133]

[0134] in, This represents the image brightness distribution of the current frame. The information entropy of the image corresponds to the brightness distribution of the current frame and is used to measure background complexity.

[0135] Obstruction detection:

[0136]

[0137] Here, overlap_ratio represents the proportion of overlap between the target and potential occlusions. This represents the occlusion threshold. When the overlap ratio between the target and a potential occluder exceeds this threshold, the system considers the target to be occluded.

[0138] Changes in weather conditions:

[0139]

[0140] in, , , These are the weighting parameters for weather factors. This indicates the fog density in the environment at the current moment. Higher fog density will result in poorer detection capabilities of the visual sensor. This indicates the rainfall intensity at the current moment. High rainfall intensity can also affect the image quality of the camera and the performance of the infrared sensor, which may lead to a decrease in target detection accuracy. This indicates the wind speed at the current moment. High wind speed can affect the flight stability of drones, causing jitter during target tracking and affecting the accuracy of flight path planning.

[0141] The comprehensive environmental change indicators are:

[0142]

[0143] in, , , , For the weight parameters, satisfying + + + = 1.

[0144] Prediction function The neural network model trained on historical data is used, and the specific calculation process is as follows:

[0145] (1) Feature vector construction:

[0146]

[0147] Where n is the length of the historical time window.

[0148] (2) Neural network structure: The prediction function adopts a three-layer fully connected neural network:

[0149]

[0150]

[0151]

[0152] in, and These are the weight matrix and bias vector of the i-th layer, respectively.

[0153] (3) Loss function: The network training uses the binary cross-entropy loss function:

[0154]

[0155] Where y is the actual target missing label (0 indicates not lost, 1 indicates lost).

[0156] (4) Threshold setting and decision-making:

[0157] Decision = {

[0158] Adjust flight path, if >

[0159] Keep tracking, if ≤ ≤ _

[0160] Normal flight, if <

[0161] }

[0162] in, = 0.7, = 0.3 is the empirical threshold.

[0163] when When the threshold is exceeded, the system adjusts the drone's attitude or flight path according to the following strategies:

[0164] If T(t) is large and E(t) is small, prioritize adjusting the flight altitude and angle to obtain a better observation perspective;

[0165] If E(t) is large and T(t) is small, prioritize adjusting sensor parameters (such as exposure time, gain, etc.).

[0166] If both T(t) and E(t) are large, activate the emergency search mode to expand the search area and reduce the flight speed.

[0167] Step S2: Target tracking based on the DeepSORT algorithm.

[0168] Building upon target detection, the system employs the DeepSORT algorithm for continuous target tracking. DeepSORT (Deep Simple Online and Realtime Tracking) is a multi-target tracking algorithm that constructs a target's state vector by fusing bounding box information and motion information, and then uses Kalman filtering to predict the target's trajectory. Simultaneously, the Hungarian algorithm is used to match the target detection box with the tracking trajectory, ensuring target consistency across consecutive frames. Even with brief target occlusion or intersecting movements, the system maintains stable tracking performance. To further enhance tracking reliability, a target loss prediction mechanism is also designed. By analyzing target movement trends and environmental changes, the system proactively adjusts the UAV's attitude or flight path, reducing the probability of target loss.

[0169] The target state is represented as:

[0170] In DeepSORT, the state of each target is represented by a state vector. This represents the state vector, which includes information such as the target's position and velocity. For a target on a two-dimensional plane, the state vector can be represented as:

[0171]

[0172] in, Indicates the center coordinates of the target bounding box. This indicates the width and height of the target bounding box. Indicates the target is and Velocity in direction, including position information , All data are obtained through YOLOv11 detection as described in step 1. The YOLOv11 detection yields a target bounding box, whose center coordinates, width, and height are easily calculated. (Velocity information) That is, the “…” obtained below The velocity vector at time t The components in the direction and direction.

[0173] The prediction phase of Kalman filtering includes:

[0174] DeepSORT uses Kalman filtering to predict the target's state and updates the state estimate using observations. The state prediction equation is:

[0175]

[0176] in, Based on Time-state pair Prediction of the state at any given moment; It is a state transition matrix, describing how the state changes from one state to another. Evolving to time.

[0177] The error covariance prediction equation is:

[0178]

[0179] in, It is the error covariance matrix of the predicted state. It is the process noise covariance matrix.

[0180] The update phase of the Kalman filter includes:

[0181] Kalman gain The calculation formula is:

[0182]

[0183] in, It is the Kalman gain; It is the observation matrix, which describes how the state is mapped to the observations; It is the observation noise covariance matrix.

[0184] The formula for state update is:

[0185]

[0186] in, This is the updated state estimate. It is the observation value at the current moment.

[0187] The formula for updating the error covariance is:

[0188]

[0189] in, It is an identity matrix.

[0190] Target matching includes:

[0191] DeepSORT uses the Hungarian algorithm to match target detection results with existing tracking trajectories. The matching process is based on two costs:

[0192] (1) Motion cost: Mahalanobis distance between the state predicted by Kalman filter and the detection result:

[0193]

[0194] in, It is the first One test result, It is the first The predicted state of a tracking trajectory. This is the prediction error covariance matrix of the Kalman filter. H refers to the preceding observation matrix. .

[0195] (2) Appearance cost: based on the cosine similarity between appearance features of the target (such as ReID features):

[0196]

[0197] in, , They are the first The tracking trajectory and the first The appearance feature vector of each detection result.

[0198] The final matching cost is a weighted sum of the movement cost and the appearance cost:

[0199]

[0200] in It is a weighting parameter used to balance the importance of motion information and appearance information.

[0201] When a target is briefly lost, DeepSORT retains the target's tracking trajectory and attempts to rematch it in subsequent frames. If the target fails to match successfully for an extended period, it is marked as lost and removed from the tracking list.

[0202] DeepSORT outputs the tracking trajectory for each target, including information such as the target's position, velocity, and appearance features. The tracking trajectory can be represented as:

[0203]

[0204] in It is the first The goal is at a certain moment State estimation.

[0205] DeepSORT uses Kalman filtering to predict target state, Hungarian algorithm to match detection results with tracking trajectory, and combines appearance features to improve matching accuracy, enabling efficient and stable multi-target tracking.

[0206] Step S3: Intelligent planning of UAV flight paths based on a high-precision digital surface model (DSM) and an improved A* algorithm.

[0207] To support intelligent flight path planning for UAVs, the system integrates a high-precision digital surface model (DSM). The DSM, constructed by fusing remote sensing imagery and LiDAR point cloud data, provides three-dimensional terrain information of the target area. During flight path planning, the system first predicts the target's future direction of motion based on the target position and trajectory output by the target tracking module. The specific method is as follows:

[0208] Based on the target tracking trajectory output by the DeepSORT algorithm in step S2 The system uses multiple methods to predict the target's future direction of motion:

[0209] Using the target's position information over the past n time steps, the least squares method is used to fit the motion trajectory:

[0210]

[0211] in, The velocity vector at time t is calculated from the historical position:

[0212]

[0213] Combining the Kalman filter from step S2, predict the target's position k steps in the future:

[0214]

[0215] in, Here is the state transition matrix. This is a state estimate for the current moment.

[0216] The system analyzes the target's motion patterns (linear motion, curvilinear motion, periodic motion, etc.) and selects the corresponding prediction model. For linear motion, a constant velocity model is used for prediction; for curvilinear motion, a constant acceleration model or polynomial fitting is used; and for periodic motion, Fourier analysis is used to predict the periodic trajectory.

[0217] Motion pattern recognition calculates the curvature of a trajectory. To achieve:

[0218]

[0219] The system predicts the future location of the target based on the identified motion patterns. , , , These represent the instantaneous velocity of the target in the x-direction, the instantaneous acceleration of the target in the y-direction, the instantaneous velocity of the target in the y-direction, and the instantaneous acceleration of the target in the x-direction, respectively.

[0220] Subsequently, combining the terrain elevation data provided by DSM, an improved A* algorithm was used to plan the UAV's flight path. The improved A* algorithm introduces terrain constraints and dynamic obstacle avoidance mechanisms on the basis of the traditional algorithm, which can generate a smooth and safe flight path, ensuring that the UAV maintains continuous tracking of the target in complex terrain environments.

[0221] The construction of the DSM includes:

[0222] A Digital Surface Model (DSM) is a three-dimensional model representing the elevation information of the Earth's surface and objects on it (such as buildings, trees, etc.). In UAV target tracking and intelligent flight path planning, DSMs provide crucial terrain and environmental information, helping UAVs to navigate accurately and avoid obstacles in complex environments. DSMs can be constructed by fusing remote sensing imagery and LiDAR point cloud data. Assuming the LiDAR point cloud data is... , where each point Represents a point in three-dimensional space. It is a planar coordinate system. It is its high-level information.

[0223] Discrete point cloud data can be converted into a continuous point cloud model (DSM) using inverse distance weighted interpolation or kriging interpolation algorithms. A DSM can be represented as a two-dimensional function:

[0224]

[0225] in, Represents plane coordinates The surface elevation at that location.

[0226] In drone flight path planning, DSM provides terrain elevation information to help drones avoid obstacles and plan smooth flight paths.

[0227] DSM provides terrain elevation information. Drones need to meet altitude constraints during flight:

[0228]

[0229] in, It is the target location output by DeepSORT as described in step S2. It is the terrain elevation provided by DSM. It is the safe altitude between the drone and the ground. and These are the minimum and maximum altitude limits for drone flight;

[0230] DSM can also be used to detect dynamic obstacles (such as buildings, trees, etc.). Assuming the obstacle's height is... Drones need to meet obstacle avoidance constraints:

[0231]

[0232] If an obstacle is detected, the system should replan the route to avoid it.

[0233] Route planning based on the improved A* algorithm includes:

[0234] In flight path planning, the system employs an improved A* algorithm, combined with terrain information provided by the DSM (Distributed Mapping System), to generate a smooth and safe flight path. Let the number of nodes in the UAV flight path be... The cost function of the A* algorithm Defined as:

[0235]

[0236] in, From the starting point to the node The actual cost, From node Heuristic cost estimation to the target point;

[0237] Among them, the cost letter and heuristic functions Improvements were made by combining terrain elevation information and dynamic obstacle information:

[0238]

[0239]

[0240] in, It is a node To the node The planar distance; It is a node To the node The change in altitude; It is a node The cost of obstacles (if there are obstacles) (If it is infinity, otherwise it is 0). , , It is a weighting parameter used to balance distance, height variation, and obstacle cost; These are the coordinates of the target point; It is a node The coordinates.

[0241] Step S4: Dynamic adjustment of flight path planning based on reinforcement learning (RL).

[0242] After the flight path is planned, the system sends the planned path to the UAV control module. The control module generates flight commands based on the path information, driving the UAV to fly along the planned route. Simultaneously, the system monitors the target's motion status and environmental changes in real time, dynamically adjusting the flight path to cope with sudden changes in the target's trajectory or the addition of environmental obstacles. When the target is lost or the UAV encounters unforeseen obstacles, the system automatically activates search mode, using the YOLO11 model to scan the target area and combining it with DSM data to replan the flight path until the target is re-locked or the mission is completed.

[0243] Reinforcement learning learns optimal flight path adjustment strategies through interaction with the environment. For a drone, acting as an intelligent agent, the environment includes the target location, terrain information (DSM), and obstacle distribution. The core of reinforcement learning is defining a reward function; the drone learns the optimal strategy by maximizing the accumulated reward. Let the coordinates of the target point be... Drones at the nodes The original planned coordinates were Therefore, in this invention, the state space of reinforcement learning It can be represented as:

[0244]

[0245] in, It is the set of locations of obstacles. It is the terrain and elevation information of the current area.

[0246] State space of reinforcement learning It can be represented as:

[0247]

[0248] in, , , respectively drones in , , Change in velocity in a direction.

[0249] State space of reinforcement learning It can be represented as:

[0250]

[0251] in, It is a drone at the node The smaller the distance between the original position and the target, the higher the reward; CollisionCost is the collision cost, which increases the penalty if the drone approaches obstacles or terrain; EnergyCost is the energy cost, which is related to the drone's speed and flight time. , , It is a weighting parameter used to balance distance, collision, and energy consumption.

[0252] Through reinforcement learning algorithms (such as Deep Q-Network, DQN), drones can learn a policy. This maximizes the cumulative reward, starting from the current node. Initially, the subsequent drone path is optimal:

[0253]

[0254] in It is a discount factor. It refers to the time frame of the task.

[0255] Through the specific implementation methods described above, this invention enables UAVs to perform target tracking and intelligent route planning in complex environments, significantly improving the autonomy and adaptability of UAVs in target tracking tasks. Experimental results show that this system outperforms existing technologies in terms of target detection accuracy, tracking stability, and route planning efficiency, and has broad application prospects.

[0256] The implementation of the various embodiments of the present invention is based on programmed processing by a device with processor functionality. Therefore, in practical engineering, the technical solutions and functions of the various embodiments of the present invention are encapsulated into various modules. Based on this reality, and building upon the above embodiments, the embodiments of the present invention provide a UAV target tracking and intelligent route planning system based on YOLO and DSM. This system is used to execute the UAV target tracking and intelligent route planning method based on YOLO and DSM in the above method embodiments.

[0257] The system includes: a target detection module, which acquires video streams, thermal imaging, and distance information in real time using multimodal sensors, fuses the multimodal data using Kalman filtering, and detects targets using the YOLOv11 model, outputting the target's position, category, and confidence information; a target tracking module, which continuously tracks targets using the DeepSORT algorithm, predicts target trajectories using Kalman filtering, and matches the detected and tracked trajectories using the Hungarian algorithm; a path planning module, which plans the UAV's flight path using terrain elevation information provided by the DSM and an improved A* algorithm; and a dynamic adjustment module, which dynamically adjusts the flight path planning based on the target's motion state and environmental changes using a reinforcement learning algorithm.

[0258] The UAV target tracking and intelligent route planning system based on YOLO and DSM provided in this embodiment of the invention addresses the current limitations in target tracking and route planning in complex environments. By employing the aforementioned modules and combining the YOLOv11 target detection algorithm, DeepSORT target tracking algorithm, high-precision digital surface model (DSM), and improved A* algorithm, it can effectively address the target tracking and route planning problems in complex environments, significantly improving the autonomy and adaptability of UAVs.

[0259] It should be noted that the system embodiments provided by the present invention are used not only to implement the methods in the above method embodiments, but also to implement the methods in other method embodiments provided by the present invention. The only difference is that corresponding functional modules are set. The principle is basically the same as that of the above system embodiments provided by the present invention. As long as those skilled in the art can improve the modules in the above system embodiments by referring to the specific technical solutions in other method embodiments and combining technical features to obtain corresponding technical means and technical solutions composed of these technical means, on the basis of the above system embodiments, and on the premise of ensuring the practicality of the technical solutions, they can obtain corresponding system-like embodiments for implementing the methods in other method-like embodiments.

[0260] Based on the same inventive concept as the foregoing embodiments, this embodiment of the invention also provides a non-transitory computer-readable storage medium storing computer instructions. These computer instructions instruct the computer to execute the UAV target tracking and intelligent flight path planning method based on YOLO and DSM, with the following steps:

[0261] The system acquires video streams, thermal imaging, and distance information in real time using multimodal sensors, fuses the multimodal data using Kalman filtering, and detects targets using the YOLOv11 model, outputting the target's location, category, and confidence information.

[0262] The DeepSORT algorithm is used to continuously track the target, combined with Kalman filtering to predict the target's trajectory, and the Hungarian algorithm is used to match the target detection and tracking trajectory.

[0263] The flight path of the UAV is planned by using the terrain elevation information provided by DSM and combining it with the improved A* algorithm.

[0264] Based on reinforcement learning algorithms, the flight path planning is dynamically adjusted according to the target's motion state and environmental changes.

[0265] In summary, the present invention discloses a method for UAV target tracking and intelligent route planning based on YOLOv11 and Digital Surface Model (DSM), which belongs to the field of UAV technology. This method combines deep learning target detection algorithms with high-precision geospatial data to achieve accurate target tracking and autonomous navigation of UAVs in complex environments. Specifically, it includes the following steps: The UAV uses multimodal sensors (such as high-definition cameras, infrared sensors, and radar) to collect video streams, thermal imaging, and distance information in real time. Kalman filtering is used to fuse the multimodal data, and the YOLOv11 model is used to efficiently detect targets, outputting the target's position, category, and confidence information. The DeepSORT algorithm is used for continuous target tracking, combined with Kalman filtering to predict the target's trajectory, and the Hungarian algorithm is used to match the target detection and tracking trajectory, ensuring stable target tracking in complex environments. Terrain elevation information provided by the DSM (Digital Sounding System) is used, combined with an improved A* algorithm, to plan the UAV's flight path, ensuring that the UAV avoids obstacles and maintains continuous target tracking in complex terrain environments. Through reinforcement learning algorithms, the UAV can dynamically adjust its flight path planning based on the target's motion state and environmental changes, ensuring that the system can automatically replan the flight path when the target is lost or environmental obstacles are added. This invention significantly improves the autonomy and adaptability of UAVs in target tracking tasks and has broad application prospects.

[0266] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A method for UAV target tracking and intelligent route planning based on YOLO and DSM, characterized in that, include: Video streams, thermal imaging, and distance information are acquired in real time using multimodal sensors. Kalman filtering is used to fuse the multimodal data, and a YOLOv11 model is used to detect targets, outputting their location, category, and confidence level. The YOLOv11 model incorporates an adaptive model update mechanism to dynamically adjust model weights based on changes in the target's appearance. This adaptive model update mechanism is as follows: ， in, Indicates the current model weights. For learning rate, The gradient of the loss function; the model weights are updated in real time by dynamically adjusting the learning rate. for: ， in, The initial learning rate, For sensitivity parameters to changes in appearance, For stability decay parameters, For the duration of stable target tracking, To comprehensively evaluate changes in appearance; The DeepSORT algorithm is used to continuously track the target, combined with Kalman filtering to predict the target's trajectory, and the Hungarian algorithm is used to match the target detection and tracking trajectory. The flight path of the UAV is planned by using the terrain elevation information provided by DSM and combining it with the improved A* algorithm. Based on reinforcement learning algorithms, the flight path planning is dynamically adjusted according to the target's motion state and environmental changes.

2. The UAV target tracking and intelligent route planning method based on YOLO and DSM according to claim 1, characterized in that, The YOLOv11 model introduces an attention mechanism and lightweight design to optimize the network structure, while FocalLoss is used to optimize the loss function. A target loss prediction mechanism is introduced to estimate the probability of target loss by analyzing target movement trends and environmental changes.

3. The UAV target tracking and intelligent route planning method based on YOLO and DSM according to claim 2, characterized in that, Introducing a target loss prediction mechanism ， in, For the prediction function, For the target movement trend, Due to environmental change, This represents the probability of target loss.

4. The UAV target tracking and intelligent route planning method based on YOLO and DSM according to claim 3, characterized in that, The prediction function is implemented using a neural network model trained on historical data, and the neural network model adopts a three-layer fully connected neural network structure.

5. The UAV target tracking and intelligent route planning method based on YOLO and DSM according to claim 3, characterized in that, The target motion trend includes: the motion speed change trend, the motion direction change trend, the motion trajectory prediction error, and the target bounding box change trend; The environmental changes include: changes in lighting, changes in background complexity, detection of obstructions, and changes in weather conditions.

6. The UAV target tracking and intelligent route planning method based on YOLO and DSM according to claim 1, characterized in that, Using terrain elevation information provided by DSM, and combining it with an improved A* algorithm, the flight path of the UAV is planned, including: Let the number of nodes in the drone flight path be... The cost function of the A* algorithm Defined as: ， in, From the starting point to the node The actual cost, From node Heuristic cost estimation to the target point; Among them, the cost letter and heuristic functions Improvements were made by combining terrain elevation information and dynamic obstacle information: ， ， in, It is a node To the node The planar distance; It is a node To the node The change in altitude; It is a node The cost of obstacles; , , It is a weighting parameter used to balance distance, height variation, and obstacle cost; These are the coordinates of the target point; It is a node The coordinates.

7. A UAV target tracking and intelligent flight path planning system based on YOLO and DSM, characterized in that, include: The target detection module is used to acquire video streams, thermal imaging, and distance information in real time through multimodal sensors, fuse the multimodal data using Kalman filtering, and detect targets using a YOLOv11 model, outputting the target's location, category, and confidence information. The YOLOv11 model incorporates an adaptive model update mechanism to dynamically adjust model weights based on changes in the target's appearance. The introduced adaptive model update mechanism is as follows: ， in, Indicates the current model weights. For learning rate, The gradient of the loss function; the model weights are updated in real time by dynamically adjusting the learning rate. for: ， in, The initial learning rate, For sensitivity parameters to changes in appearance, For stability decay parameters, For the duration of stable target tracking, To comprehensively evaluate changes in appearance; The target tracking module is used to continuously track the target using the DeepSORT algorithm, combined with Kalman filtering to predict the target's trajectory, and uses the Hungarian algorithm to match the target detection with the tracking trajectory. The path planning module is used to plan the flight path of the UAV using the terrain elevation information provided by DSM and the improved A* algorithm. The dynamic adjustment module is used to dynamically adjust the flight path planning based on the target's motion state and environmental changes, using reinforcement learning algorithms.

8. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions that cause the computer to execute the UAV target tracking and intelligent route planning method based on YOLO and DSM as described in any one of claims 1 to 6.

Images