Unmanned aerial vehicle target tracking and intelligent route planning method and system based on YOLO and DSM

Abstract

The invention discloses an unmanned aerial vehicle target tracking and intelligent route planning method based on YOLO and DSM, and the method comprises the steps: enabling an unmanned aerial vehicle to collect video streaming, thermal imaging and distance information in real time through a multi-modal sensor, fusing the multi-modal data through Kalman filtering, detecting a target through a YOLOv11 model, and outputting the position, category and confidence information of the target; a DeepSORT algorithm is adopted to continuously track a target, Kalman filtering is combined to predict a target motion trajectory, and a Hungary algorithm is adopted to realize matching of target detection and a tracking trajectory, so that stable tracking in a complex environment is ensured; the flight path of the unmanned aerial vehicle is planned by using terrain elevation information provided by the DSM and combining with an improved A * algorithm, so that the unmanned aerial vehicle is ensured to avoid obstacles and keep continuous tracking of a target in a complex terrain environment; through a reinforcement learning algorithm, the unmanned aerial vehicle can dynamically adjust the route planning according to the motion state of the target and the environment change, and it is ensured that the system can automatically re-plan the route when the target is lost or the environment obstacle is newly added.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to a UAV target tracking and intelligent route planning method based on the YOLO11 target detection algorithm and a digital surface model (DSM). Background Art

[0002] With the rapid development of drone technology, drones are increasingly being used in target tracking, environmental monitoring, disaster relief, and other fields. However, target tracking and route planning in complex environments still face many challenges:

[0003] 1. In dynamic environments, the appearance, posture, and trajectory of a target may change dramatically. Traditional target detection and tracking algorithms struggle to cope with issues such as target loss and occlusion in complex scenarios.

[0004] 2. Drones are typically equipped with multiple sensors (such as cameras, infrared sensors, and radars). Efficiently fusing multimodal data to improve the robustness of target detection and tracking remains a technical challenge.

[0005] 3. Route planning in complex terrain: In complex terrain, drones need to avoid obstacles (such as buildings and trees) and maintain continuous target tracking. Traditional route planning algorithms struggle to balance the multiple requirements of terrain constraints, dynamic obstacle avoidance, and target tracking.

[0006] 4. During flight, drones may encounter unexpected situations such as target loss and new environmental obstacles. Traditional route planning methods lack sufficient adaptive capabilities and it is difficult to adjust the flight path in real time.

[0007] Several existing solutions exist to address these issues, including deep learning-based target detection algorithms (such as the YOLO series), Kalman filter-based target tracking algorithms, and A*-based route planning methods. However, these methods still have limitations in target tracking and route planning in complex environments, particularly in multimodal data fusion, dynamic obstacle avoidance, and adaptive route adjustment. Summary of the Invention

[0008] To overcome the shortcomings of the above-mentioned existing technologies, the present invention provides a UAV target tracking and intelligent route planning method based on YOLO and DSM. By combining the YOLOv11 target detection algorithm, the DeepSORT target tracking algorithm, the high-precision digital surface model (DSM) and the improved A* algorithm, the present invention can effectively cope with target tracking and route planning problems in complex environments, and significantly improve the autonomy and adaptability of the UAV.

[0009] According to one aspect of the present invention, a method for tracking and intelligent route planning of a drone based on YOLO and DSM is provided, comprising:

[0010] Multimodal sensors collect video streams, thermal imaging, and distance information in real time, fuse the multimodal data using Kalman filtering, and detect targets using the YOLOv11 model, outputting the target's location, category, and confidence level.

[0011] The DeepSORT algorithm is used to continuously track the target, combined with the Kalman filter to predict the target trajectory, and the Hungarian algorithm is used to match the target detection with the tracking trajectory;

[0012] The terrain elevation information provided by DSM is used in combination with the improved A* algorithm to plan the flight path of the UAV;

[0013] Based on the reinforcement learning algorithm, the route planning is dynamically adjusted according to the target's motion state and environmental changes.

[0014] As a further technical solution, the YOLOv11 model introduces an attention mechanism and lightweight design to optimize the network structure, and uses Focal Loss to optimize the loss function;

[0015] Introducing an adaptive model update mechanism to dynamically adjust model weights according to changes in target appearance;

[0016] A target loss prediction mechanism is introduced to estimate the target loss probability by analyzing the target motion trend and environmental changes.

[0017] As a further technical solution, an adaptive model update mechanism is introduced as

[0018]

[0019] in, represents the current model weight, is the learning rate, is the gradient of the loss function.

[0020] As a further technical solution, the model weights are updated in real time by dynamically adjusting the learning rate. for:

[0021]

[0022] in, is the initial learning rate, is the appearance change sensitivity parameter, is the stability decay parameter, is the duration of stable target tracking, It is a comprehensive appearance change indicator.

[0023] As a further technical solution, the target loss prediction mechanism is introduced as

[0024]

[0025] in, is the prediction function, For the target movement trend, For environmental changes, is the target loss probability.

[0026] As a further technical solution, the prediction function is implemented using a neural network model trained based on historical data, and the neural network model adopts a three-layer fully connected neural network structure.

[0027] As a further technical solution, the target motion trend includes: motion speed change trend, motion direction change trend, motion trajectory prediction error and target bounding box change trend;

[0028] The environmental changes include: changes in illumination, changes in background complexity, detection of obstructions, and changes in weather conditions.

[0029] As a further technical solution, the terrain elevation information provided by DSM is combined with an improved A* algorithm to plan the flight path of the UAV, including:

[0030] Assume the number of nodes in the drone route is , the cost function of the A* algorithm Defined as:

[0031]

[0032] in, From the starting point to the node The actual cost, It is a slave node Heuristic estimate of cost to the target point;

[0033] Among them, the consideration letter and the heuristic function Improved by combining terrain elevation information and dynamic obstacle information:

[0034]

[0035]

[0036] in, is a node To Node Plane distance; is a node To Node height changes; is a node The cost of obstacles; 、 、 is a weight parameter used to balance distance, height change, and obstacle cost; are the coordinates of the target point; is a node 's coordinates.

[0037] According to one aspect of the present invention, a UAV target tracking and intelligent route planning system based on YOLO and DSM is provided, comprising:

[0038] The target detection module is used to collect video streams, thermal images, and distance information in real time through multimodal sensors, fuse multimodal data using Kalman filtering, detect targets using the YOLOv11 model, and output the target's location, category, and confidence information;

[0039] The target tracking module uses the DeepSORT algorithm to continuously track the target, combines it with the Kalman filter to predict the target's motion trajectory, and uses the Hungarian algorithm to match target detection with the tracking trajectory;

[0040] The path planning module is used to plan the flight path of the UAV using the terrain elevation information provided by the DSM and the improved A* algorithm;

[0041] The dynamic adjustment module is used to dynamically adjust the route planning according to the target's motion state and environmental changes based on the reinforcement learning algorithm.

[0042] According to one aspect of the present invention, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to execute the YOLO and DSM-based drone target tracking and intelligent route planning method.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] 1. This paper proposes a new UAV target tracking and intelligent route planning method by combining the YOLOv11 target detection algorithm, the DeepSORT target tracking algorithm, a high-precision digital surface model (DSM), and an improved A* algorithm. This method can effectively address target tracking and route planning problems in complex environments, significantly improving the autonomy and adaptability of UAVs.

[0045] 2. The method of the present invention achieves accurate tracking of targets and autonomous navigation of UAVs in complex environments by combining deep learning target detection algorithms with high-precision geospatial data. It can significantly improve the autonomy and adaptability of UAVs in target tracking tasks and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, a brief introduction will be given below to the drawings used in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 A flowchart of a method for drone target tracking and intelligent route planning based on YOLO and DSM is provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] Unless otherwise defined, the technical or scientific terms used herein shall have the ordinary meaning understood by persons of ordinary skill in the field to which the present disclosure belongs. The words "first", "second", "third" and similar terms used in the patent application specification and claims of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one" or "a" do not indicate a quantity limitation, but rather indicate the existence of at least one. Words such as "include" or "comprising" mean that the elements or objects appearing before "include" or "comprising" cover the elements or objects listed after "include" or "comprising" and their equivalents, and do not exclude other elements or objects. Words such as "connected" or "connected" and similar terms are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0049] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. In addition, the technical features in the various embodiments or single embodiments provided by the present invention are arbitrarily combined with each other to form a new technical solution. This combination is not restricted by the sequence of steps and / or structural composition mode, but must be based on the ability of ordinary technicians in this field to implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that this combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0050] This paper provides a method for drone target tracking and intelligent route planning based on YOLO11 and digital surface model. Its core lies in combining deep learning target detection algorithm with high-precision geospatial data to achieve accurate target tracking and autonomous navigation of drones in complex environments. Figure 1 , the specific implementation method is described in detail below.

[0051] Step S1, target detection based on fused data of YOLO11.

[0052] First, the drone collects video stream data, thermal imaging data, and distance information in real time through its multimodal sensors, including high-definition cameras, infrared sensors, and radars. It then fuses this data and transmits it to the onboard computing unit:

[0053]

[0054] in, Represents the video stream data collected by the high-definition camera, Represents the thermal imaging data collected by the infrared sensor, Indicates the distance information collected by the radar, including various information such as the observation value and observation noise described below.

[0055] Among them, the system model is:

[0056] Through the Kalman filter fusion algorithm, multimodal data is fused into a unified input. Assume that the system has sensors, each sensor provides different observation data. , the state vector of the system is , the state transition matrix (describing how the system state changes over time) is , the process noise is (It is usually assumed to be zero-mean Gaussian noise, and the corresponding covariance matrix is ), then the state equation of the system can be expressed as:

[0057]

[0058] The observation model is:

[0059] At the moment , for different sensors (such as cameras, infrared sensors, radars, etc.), indivual The sensor observation value is recorded as , will indivual The observation matrix corresponding to the sensor (describing how to extract the observation value from the state vector) is recorded as , will indivual The observation noise corresponding to the sensor (usually assumed to be zero-mean Gaussian noise, the corresponding covariance matrix is ) is recorded as , then the observation equation of each sensor is:

[0060]

[0061] in, It is The observation noise of the sensors, the covariance matrix is ;

[0062] The multimodal data fusion of Kalman filter is:

[0063] In the prediction stage, the system estimates the state of the previous moment and covariance matrix , predict the current state and covariance matrix :

[0064]

[0065]

[0066] In the update phase, the system uses the observations of each sensor , gradually update the state estimate and covariance matrix. sensors, Kalman gain The calculation formula is:

[0067]

[0068] Then, use the Observations from sensors Update state estimate and the covariance matrix :

[0069]

[0070]

[0071] in, is the identity matrix.

[0072] For multiple sensors, the observation value of each sensor can be used to update in turn. The update process can be expressed as:

[0073]

[0074]

[0075] for arrive , update the following in sequence:

[0076]

[0077]

[0078]

[0079] Finally, after all sensors are updated, the fused state estimate is obtained and the covariance matrix :

[0080]

[0081]

[0082] Through the above steps, the Kalman filter can fuse observation data from multiple sensors to obtain a more accurate state estimate. The core idea of ​​multimodal data fusion is to use the complementary information from different sensors and gradually correct the state estimate through the prediction and update mechanism of the Kalman filter, thereby improving the robustness and accuracy of the system.

[0083] The video frame of the data processed by multimodal fusion is input into the target detection network based on YOLO11 (the "video frame" here is the data processed by multimodal fusion The YOLO11 model significantly improves the accuracy and speed of object detection by optimizing the network structure (such as the introduction of an attention mechanism and lightweight design) and loss function (such as the use of Focal Loss to address class imbalance), making it particularly suitable for real-time detection of drones in dynamic environments. Furthermore, the system incorporates an adaptive model update mechanism that dynamically adjusts model weights based on changes in target appearance (such as lighting conditions and target pose), further enhancing detection robustness.

[0084] The output of the YOLO11 model can be expressed as:

[0085]

[0086] in, represents the center coordinates of the target bounding box, represents the width and height of the bounding box, represents the target category, Represents the target confidence.

[0087] YOLO11 can significantly improve detection efficiency by introducing attention mechanisms (such as SE modules) and lightweight designs (such as depthwise separable convolutions). Its loss function uses Focal Loss and is defined as:

[0088]

[0089] in, represents the target probability predicted by the model, and is a hyperparameter used to address the class imbalance problem.

[0090] The adaptive model update mechanism dynamically adjusts the model weights , to adapt to the changes in target appearance. The update formula is:

[0091]

[0092] in, represents the current model weight, is the learning rate, is the gradient of the loss function. By dynamically adjusting the learning rate To update the model weights in real time, the system can maintain high detection accuracy when the target appearance changes (such as lighting changes, target posture changes).

[0093] Dynamically adjust the learning rate when the target appearance changes (such as lighting changes, target pose changes) The specific methods are as follows:

[0094] The method for detecting light changes is:

[0095]

[0096] in, Indicates the pixel intensity of the target area in the current frame, represents the pixel intensity of the target area in the reference frame, The meaning of the function is to calculate the average brightness of the image.

[0097] The detection method for posture changes is:

[0098]

[0099] in, and Represent the target bounding box parameters of the current frame and the reference frame respectively.

[0100] The method for detecting feature similarity changes is:

[0101]

[0102] in, and represent the depth feature vectors extracted from the current frame and the reference frame, respectively.

[0103] The comprehensive appearance change indicators are:

[0104]

[0105] Among them, w1, w2, and w3 are weight parameters, satisfying .

[0106] According to the degree of change in the integrated appearance, the system dynamically adjusts the learning rate:

[0107]

[0108] in, is the initial learning rate, is the appearance change sensitivity parameter, is the stability decay parameter, The duration of stable tracking of the target.

[0109] In order to further improve the robustness of target detection and tracking, a target loss prediction mechanism is introduced into the system. and environmental changes To estimate the target loss probability :

[0110]

[0111] in, is a prediction function used to calculate the target loss probability. When the threshold is exceeded, the system adjusts the drone's attitude or route in advance to reduce the probability of target loss.

[0112] The target movement trend T is quantitatively analyzed through the following multiple dimensions:

[0113] Movement speed change trend:

[0114]

[0115] in, Indicates that the target is at time The movement speed is calculated by the target position change between consecutive frames.

[0116] Movement direction change trend:

[0117]

[0118] in, Indicates that the target is at time The angle of the direction of movement.

[0119] Motion trajectory prediction error:

[0120]

[0121] Among them, pos_predicted(t) is the target position predicted by the Kalman filter, and pos_actual(t) is the actual detected target position.

[0122] Target bounding box change trend:

[0123]

[0124] in, Represents the area of ​​the target bounding box.

[0125] The comprehensive movement trend indicators are:

[0126]

[0127] in, 、 、 、 is the weight parameter, satisfying + + + = 1.

[0128] Environmental change E is quantitatively assessed using the following factors:

[0129] Lighting changes:

[0130]

[0131] in, Indicates the image brightness distribution of the current frame.

[0132] Background complexity changes:

[0133]

[0134] in, Indicates the image brightness distribution of the current frame, Represents the information entropy of the image corresponding to the image brightness distribution of the current frame, which is used to measure the background complexity.

[0135] Occlusion detection:

[0136]

[0137] Among them, overlap_ratio represents the overlap ratio between the target and the potential occluder. Represents the occlusion threshold. When the overlap ratio between the target and the potential occluder exceeds this threshold, the system considers the target to be occluded.

[0138] Changing weather conditions:

[0139]

[0140] in, 、 、 is the weather factor weight parameter. Indicates the fog density in the environment at the current moment. The higher the fog density, the worse the detection ability of the visual sensor. Indicates the current rainfall intensity. High rainfall intensity will also affect the camera image quality and the performance of the infrared sensor, which may lead to a decrease in target detection accuracy. Indicates the current wind speed. High wind speed will affect the flight stability of the drone, causing jitter during target tracking and affecting the accuracy of route planning.

[0141] The comprehensive environmental change indicators are:

[0142]

[0143] in, 、 、 、 is the weight parameter, satisfying + + + = 1.

[0144] Prediction Function Using a neural network model trained based on historical data, the specific calculation process is as follows:

[0145] (1) Feature vector construction:

[0146]

[0147] Where n is the length of the historical time window.

[0148] (2) Neural network structure: The prediction function uses a three-layer fully connected neural network:

[0149]

[0150]

[0151]

[0152] in, and are the weight matrix and bias vector of the i-th layer respectively.

[0153] (3) Loss function: Network training uses the binary cross entropy loss function:

[0154]

[0155] Among them, y is the true target loss label (0 means not lost, 1 means lost).

[0156] (4) Threshold setting and decision-making:

[0157] Decision = {

[0158] Adjust the route, if >

[0159] Keep track if ≤ ≤ _

[0160] Normal flight, if <

[0161] }

[0162] in, = 0.7, = 0.3 is the empirical threshold.

[0163] when When the threshold is exceeded, the system adjusts the drone's attitude or route according to the following strategies:

[0164] If T(t) is large and E(t) is small, prioritize adjusting the flight altitude and angle to obtain a better observation angle;

[0165] If E(t) is large and T(t) is small, give priority to adjusting sensor parameters (such as exposure time, gain, etc.);

[0166] If both T(t) and E(t) are large, activate the emergency search mode, expand the search range and reduce the flight speed.

[0167] Step S2: target tracking based on DeepSORT algorithm.

[0168] Based on target detection, the system uses the DeepSORT algorithm to continuously track the target. DeepSORT (Deep Simple Online and Realtime Tracking) is a multi-target tracking algorithm that constructs the target's state vector by fusing the target's bounding box information and motion information, and uses Kalman filtering to predict the target's motion trajectory. At the same time, the Hungarian algorithm is used to match the target detection box with the tracking trajectory, ensuring the consistency of the target in consecutive frames. Even in the case of brief occlusion or cross-motion of the target, the system can still maintain stable tracking performance. To further improve tracking reliability, the system has also designed a target loss prediction mechanism. By analyzing the target's motion trends and environmental changes, it can adjust the drone's attitude or route in advance to reduce the probability of target loss.

[0169] Among them, the target state is expressed as:

[0170] In DeepSORT, the state of each target is represented by a state vector Represents, which contains information such as the position and speed of the target. For a target on a two-dimensional plane, the state vector can be expressed as:

[0171]

[0172] in, represents the center coordinates of the target bounding box, represents the width and height of the target bounding box, Indicates the target is and Speed ​​in direction, including position information 、 All of them are obtained by YOLO11 detection in step 1. What YOLO11 detects is a target bounding box, and its center coordinates, width and height are easy to obtain. Speed ​​information This is the " is the velocity vector at time t” Components in the and directions.

[0173] The prediction stage of Kalman filtering includes:

[0174] DeepSORT uses Kalman filtering to predict the state of the target and updates the state estimate through observations. The state prediction equation is:

[0175]

[0176] in, is based on Time state pair Prediction of momentary status; is the state transition matrix, describing how the state changes from Evolving to time.

[0177] The error covariance prediction equation is:

[0178]

[0179] in, is the error covariance matrix of the predicted state, is the process noise covariance matrix.

[0180] The update phase of the Kalman filter includes:

[0181] Kalman gain The calculation formula is:

[0182]

[0183] in, is the Kalman gain; is the observation matrix, which describes how states are mapped to observations; is the observation noise covariance matrix.

[0184] The formula for status update is:

[0185]

[0186] in, is the updated state estimate, is the observed value at the current moment.

[0187] The error covariance update formula is:

[0188]

[0189] in, is the identity matrix.

[0190] Target matching, including:

[0191] DeepSORT uses the Hungarian algorithm to match target detection results with existing tracking trajectories. The matching process is based on two costs:

[0192] (1) Motion cost: Mahalanobis distance between the state predicted by Kalman filter and the detection result:

[0193]

[0194] in, It is Test results, It is The predicted state of the tracking trajectory, is the prediction error covariance matrix of the Kalman filter. H refers to the previous observation matrix .

[0195] (2) Appearance cost: based on the cosine similarity between the appearance features of the target (such as ReID features):

[0196]

[0197] in, 、 They are The tracking trajectory and The appearance feature vector of the detection result.

[0198] The final matching cost is the weighted sum of motion cost and appearance cost:

[0199]

[0200] in is a weight parameter used to balance the importance of motion information and appearance information.

[0201] When a target is lost for a short time, DeepSORT will keep the target's tracking track and try to rematch it in subsequent frames. If the target is not matched successfully for a long time, it will be marked as lost and removed from the tracking list.

[0202] The output of DeepSORT is the tracking trajectory of each target, including the target's position, speed, appearance characteristics and other information. The tracking trajectory can be expressed as:

[0203]

[0204] in It is Target at the moment state estimation.

[0205] DeepSORT uses Kalman filtering to predict target states, the Hungarian algorithm to match detection results with tracking trajectories, and combines appearance features to improve matching accuracy, enabling efficient and stable multi-target tracking.

[0206] Step S3: UAV route intelligent planning based on high-precision digital surface model (DSM) and improved A* algorithm.

[0207] To support intelligent drone route planning, the system integrates a high-precision digital surface model (DSM). Constructed by fusing remote sensing imagery and LiDAR point cloud data, the DSM provides three-dimensional terrain information about the target area. During route planning, the system first predicts the target's future direction of motion based on the target position and trajectory output by the target tracking module. The specific method is as follows:

[0208] Target tracking trajectory based on the DeepSORT algorithm output in step S2 , the system uses a variety of methods to predict the target's future movement direction:

[0209] Using the target's position information in the past n time steps, the least squares method is used to fit the motion trajectory:

[0210]

[0211] in, is the velocity vector at time t, calculated from the historical position:

[0212]

[0213] Combined with the Kalman filter in step S2, the target position in the next k steps is predicted:

[0214]

[0215] in, is the state transition matrix, is the state estimate at the current moment.

[0216] The system selects the corresponding prediction model by analyzing the target's motion pattern (linear motion, curved motion, periodic motion, etc.). The linear motion pattern is predicted using a constant velocity model, the curved motion pattern is predicted using a constant acceleration model or polynomial fitting, and the periodic motion pattern is predicted based on Fourier analysis.

[0217] Motion pattern recognition by calculating the curvature of the trajectory To achieve:

[0218]

[0219] The system predicts the final target's future position based on the identified motion pattern. 、 、 、 They represent the instantaneous velocity of the target in the x direction, the instantaneous acceleration of the target in the y direction, the instantaneous velocity of the target in the y direction, and the instantaneous acceleration of the target in the x direction respectively.

[0220] Subsequently, the improved A* algorithm, combined with the terrain elevation data provided by the DSM, was used to plan the drone's flight path. This improved A* algorithm, based on the traditional algorithm, incorporates terrain constraints and a dynamic obstacle avoidance mechanism. This algorithm generates a smooth and safe flight path, ensuring that the drone maintains continuous tracking of its target in complex terrain environments.

[0221] The construction of DSM includes:

[0222] A Digital Surface Model (DSM) is a three-dimensional model that represents the elevation information of the earth's surface and objects on it (such as buildings and trees). In drone target tracking and intelligent route planning, DSM provides key terrain and environmental information, helping drones to accurately navigate and avoid obstacles in complex environments. DSM can be constructed by fusing remote sensing images and LiDAR point cloud data. Assume that the LiDAR point cloud data is , where each point represents a point in three-dimensional space, is a plane coordinate, It is its height information.

[0223] Discrete point cloud data can be converted into continuous DSM through inverse distance weighted interpolation or Kriging interpolation algorithm. DSM can be expressed as a two-dimensional function:

[0224]

[0225] in, Expressed in plane coordinates The surface elevation at the location.

[0226] In drone route planning, DSM provides terrain elevation information to help drones avoid obstacles and plan smooth flight paths.

[0227] DSM provides terrain elevation information , the drone needs to meet the altitude constraints during flight:

[0228]

[0229] in, is the target position output by DeepSORT in step S2, is the terrain elevation provided by DSM, It is the safe height between the drone and the ground. and It is the minimum and maximum altitude limit for drone flight;

[0230] DSM can also be used to detect dynamic obstacles (such as buildings, trees, etc.). Assume that the height of the obstacle is , the drone needs to meet the obstacle avoidance constraints:

[0231]

[0232] If an obstacle is detected, the system should re-route the vehicle to avoid it.

[0233] Route planning based on the improved A* algorithm, including:

[0234] In route planning, the system uses the improved A* algorithm, combined with the terrain information provided by DSM, to generate a smooth and safe flight path. Assume that the number of nodes in the UAV route is , the cost function of the A* algorithm Defined as:

[0235]

[0236] in, From the starting point to the node The actual cost, It is a slave node Heuristic estimate of cost to the target point;

[0237] Among them, the consideration letter and the heuristic function Improved by combining terrain elevation information and dynamic obstacle information:

[0238]

[0239]

[0240] in, is a node To Node Plane distance; is a node To Node height changes; is a node The obstacle cost (if there is an obstacle, then is infinite, otherwise 0); 、 、 is a weight parameter used to balance distance, height change, and obstacle cost; are the coordinates of the target point; is a node 's coordinates.

[0241] Step S4: Dynamic adjustment of route planning based on reinforcement learning (RL).

[0242] After route planning is complete, the system sends the planned path to the drone's control module. Based on this path information, the control module generates flight commands, driving the drone along the planned route. Simultaneously, the system monitors the target's motion and environmental changes in real time, dynamically adjusting the route plan to account for sudden changes in the target's trajectory or the introduction of new obstacles. If the target is lost or the drone encounters an unforeseen obstacle, the system automatically initiates search mode, scanning the target area using the YOLO11 model and replanning the route in conjunction with DSM data until the target is re-engaged or the mission is completed.

[0243] Reinforcement learning can learn the optimal route adjustment strategy through interaction with the environment. The drone is an intelligent agent, and the environment includes the target location, terrain information (DSM), obstacle distribution, etc. The core of reinforcement learning is to define a reward function. The drone learns the optimal strategy by maximizing the cumulative reward. Let the coordinates of the target point be , UAV at the node The original planning coordinates are , then in the present invention, the state space of reinforcement learning It can be expressed as:

[0244]

[0245] in, is the set of obstacle locations, It is the terrain elevation information of the current area.

[0246] State Space for Reinforcement Learning It can be expressed as:

[0247]

[0248] in, 、 、 The drones are 、 、 Changes in speed in direction.

[0249] State Space for Reinforcement Learning It can be expressed as:

[0250]

[0251] in, The drone is at the node The distance between the original position and the target, the smaller the distance, the higher the reward; CollisionCost is the collision cost, if the drone approaches an obstacle or terrain, the penalty increases; EnergyCost is the energy consumption cost, which is related to the speed change and flight time of the drone; 、 、 is a weight parameter used to balance distance, collision and energy consumption.

[0252] Through reinforcement learning algorithms (such as Deep Q Network, DQN), the drone can learn a strategy , so that the cumulative reward is maximized, that is, from the current node Initially, the following drone path is optimal:

[0253]

[0254] in is the discount factor, is the time range of the task.

[0255] Through the above-described specific implementation, the present invention enables drones to track targets and conduct intelligent route planning in complex environments, significantly improving their autonomy and adaptability in target tracking tasks. Experimental results demonstrate that the system outperforms existing technologies in target detection accuracy, tracking stability, and route planning efficiency, demonstrating its broad application prospects.

[0256] The implementation of each embodiment of the present invention is based on programmed processing by a device with processor functionality. Therefore, in practical engineering, the technical solutions and functions of each embodiment of the present invention are encapsulated into various modules. Based on this reality, and in addition to the aforementioned embodiments, an embodiment of the present invention provides a YOLO and DSM-based drone target tracking and intelligent route planning system. This system is used to implement the YOLO and DSM-based drone target tracking and intelligent route planning method described in the aforementioned method embodiment.

[0257] The system includes: a target detection module, which is used to collect video streams, thermal imaging and distance information in real time through multimodal sensors, fuse multimodal data using Kalman filtering, detect targets through the YOLOv11 model, and output the target's position, category and confidence information; a target tracking module, which is used to continuously track the target using the DeepSORT algorithm, predict the target's motion trajectory in combination with Kalman filtering, and match target detection with tracking trajectory through the Hungarian algorithm; a path planning module, which is used to plan the UAV's flight path using the terrain elevation information provided by the DSM and the improved A* algorithm; and a dynamic adjustment module, which is used to dynamically adjust the route planning based on the target's motion state and environmental changes based on the reinforcement learning algorithm.

[0258] The YOLO and DSM-based UAV target tracking and intelligent route planning system provided by the embodiment of the present invention addresses the current situation where limitations still exist in target tracking and route planning in complex environments. By adopting the aforementioned modules and combining the YOLOv11 target detection algorithm, the DeepSORT target tracking algorithm, the high-precision digital surface model (DSM) and the improved A* algorithm, it can effectively address target tracking and route planning problems in complex environments and significantly improve the autonomy and adaptability of UAVs.

[0259] It should be noted that the system embodiments provided by the present invention are not only used to implement the methods in the above-mentioned method embodiments, but also used to implement the methods in other method embodiments provided by the present invention. The only difference lies in the setting of corresponding functional modules, and the principles thereof are basically the same as the principles of the above-mentioned system embodiments provided by the present invention. As long as those skilled in the art refer to the specific technical solutions in other method embodiments on the basis of the above-mentioned system embodiments, obtain corresponding technical means and technical solutions composed of these technical means by combining technical features, and on the premise of ensuring the practicality of the technical solutions, improve the modules in the above-mentioned system embodiments to obtain corresponding system class embodiments for implementing the methods in other method class embodiments.

[0260] Based on the same inventive concept as the aforementioned embodiment, an embodiment of the present invention further provides a non-transitory computer-readable storage medium storing computer instructions. The computer instructions cause the computer to execute the YOLO and DSM-based drone target tracking and intelligent route planning method, which includes the following steps:

[0261] Multimodal sensors collect video streams, thermal imaging, and distance information in real time, fuse the multimodal data using Kalman filtering, and detect targets using the YOLOv11 model, outputting the target's location, category, and confidence level.

[0262] The DeepSORT algorithm is used to continuously track the target, combined with the Kalman filter to predict the target trajectory, and the Hungarian algorithm is used to match the target detection with the tracking trajectory;

[0263] The terrain elevation information provided by DSM is used in combination with the improved A* algorithm to plan the flight path of the UAV;

[0264] Based on the reinforcement learning algorithm, the route planning is dynamically adjusted according to the target's motion state and environmental changes.

[0265] In summary, the present invention discloses a method for unmanned aerial vehicle (UAV) target tracking and intelligent route planning based on YOLOv11 and digital surface model (DSM), which belongs to the field of UAV technology. This method combines a deep learning target detection algorithm with high-precision geospatial data to achieve precise target tracking and autonomous navigation in complex environments. Specifically, the method includes the following steps: the drone uses multimodal sensors (such as high-definition cameras, infrared sensors, and radar) to collect video streams, thermal imaging, and distance information in real time. The multimodal data is fused using a Kalman filter, and the YOLOv11 model is used to efficiently detect the target and output the target's location, category, and confidence information. The DeepSORT algorithm is used to continuously track the target, combined with a Kalman filter to predict the target's motion trajectory. The Hungarian algorithm is used to match target detection and tracking trajectories, ensuring stable tracking of the target in complex environments. Terrain elevation information provided by the DSM is combined with an improved A* algorithm to plan the drone's flight path, ensuring that the drone avoids obstacles and maintains continuous tracking of the target in complex terrain environments. Using a reinforcement learning algorithm, the drone can dynamically adjust its route planning based on the target's motion state and environmental changes, ensuring that the system can automatically replan its route if the target is lost or new environmental obstacles appear. This method can significantly improve the autonomy and adaptability of drones in target tracking tasks and has broad application prospects.

[0266] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. The UAV target tracking and intelligent route planning method based on YOLO and DSM is characterized by: include: Multimodal sensors collect video streams, thermal imaging, and distance information in real time, fuse the multimodal data using Kalman filtering, and detect targets using the YOLOv11 model, outputting the target's location, category, and confidence level. The DeepSORT algorithm is used to continuously track the target, combined with the Kalman filter to predict the target trajectory, and the Hungarian algorithm is used to match the target detection with the tracking trajectory; The terrain elevation information provided by DSM is used in combination with the improved A* algorithm to plan the flight path of the UAV; Based on the reinforcement learning algorithm, the route planning is dynamically adjusted according to the target's motion state and environmental changes.

2. The UAV target tracking and intelligent route planning method based on YOLO and DSM according to claim 1 is characterized in that: The YOLOv11 model introduces attention mechanism and lightweight design to optimize the network structure, and uses FocalLoss to optimize the loss function; Introducing an adaptive model update mechanism to dynamically adjust model weights according to changes in target appearance; A target loss prediction mechanism is introduced to estimate the target loss probability by analyzing the target motion trend and environmental changes.

3. The UAV target tracking and intelligent route planning method based on YOLO and DSM according to claim 2 is characterized in that: Introducing the adaptive model update mechanism ， in, represents the current model weight, is the learning rate, is the gradient of the loss function.

4. The UAV target tracking and intelligent route planning method based on YOLO and DSM according to claim 3 is characterized in that: Update the model weights in real time by dynamically adjusting the learning rate. for: ， in, is the initial learning rate, is the appearance change sensitivity parameter, is the stability decay parameter, is the duration of stable target tracking, It is a comprehensive appearance change indicator.

5. The UAV target tracking and intelligent route planning method based on YOLO and DSM according to claim 2 is characterized in that: The target loss prediction mechanism is introduced as ， in, is the prediction function, For the target movement trend, For environmental changes, is the target loss probability.

6. The UAV target tracking and intelligent route planning method based on YOLO and DSM according to claim 5 is characterized in that: The prediction function is implemented using a neural network model trained based on historical data, and the neural network model adopts a three-layer fully connected neural network structure.

7. The UAV target tracking and intelligent route planning method based on YOLO and DSM according to claim 5 is characterized in that: The target motion trend includes: motion speed change trend, motion direction change trend, motion trajectory prediction error and target bounding box change trend; The environmental changes include: changes in illumination, changes in background complexity, detection of obstructions, and changes in weather conditions.

8. The UAV target tracking and intelligent route planning method based on YOLO and DSM according to claim 1 is characterized in that: The terrain elevation information provided by DSM is combined with the improved A* algorithm to plan the flight path of the drone, including: Assume the number of nodes in the drone route is , the cost function of the A* algorithm Defined as: ， in, From the starting point to the node The actual cost, It is a slave node Heuristic estimate of cost to the target point; Among them, the consideration letter and the heuristic function Improved by combining terrain elevation information and dynamic obstacle information: ， in, is a node To Node Plane distance; is a node To Node height changes; is a node The cost of obstacles; 、 、 is a weight parameter used to balance distance, height change, and obstacle cost; are the coordinates of the target point; is a node 's coordinates.

9. The UAV target tracking and intelligent route planning system based on YOLO and DSM is characterized by: include: The target detection module is used to collect video streams, thermal images, and distance information in real time through multimodal sensors, fuse multimodal data using Kalman filtering, detect targets using the YOLOv11 model, and output the target's location, category, and confidence information; The target tracking module uses the DeepSORT algorithm to continuously track the target, combines it with the Kalman filter to predict the target's motion trajectory, and uses the Hungarian algorithm to match target detection with the tracking trajectory; The path planning module is used to plan the flight path of the UAV using the terrain elevation information provided by the DSM and the improved A* algorithm; The dynamic adjustment module is used to dynamically adjust the route planning according to the target's motion state and environmental changes based on the reinforcement learning algorithm.

10. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium stores computer instructions, which enable the computer to execute the drone target tracking and intelligent route planning method based on YOLO and DSM as described in any one of claims 1 to 8.

Images