A method and device for optimizing a task trajectory of a UAV based on deep learning

Abstract

The application discloses a kind of unmanned aerial vehicle task trajectory optimization method and device based on deep learning, according to the task execution signal response call and parse dispatch task, obtain task scheduling information, according to task scheduling information collection environment information of unmanned aerial vehicle to be scheduled, according to environment information and task scheduling information generate initial task trajectory, and the above two kinds of information and the environment prediction information obtained based on environment information prediction are used as the optimization parameter of deep learning model, and initial task trajectory is optimized, and optimization task trajectory is sent to unmanned aerial vehicle, to make unmanned aerial vehicle execute dispatch task according to optimization task trajectory and feedback task trajectory information. Optimal running trajectory of unmanned aerial vehicle is obtained by interactive learning with environment, and the flight speed and height of unmanned aerial vehicle are adjusted by environment prediction data, to ensure the safety of unmanned aerial vehicle task execution;According to real-time acquisition data, trajectory is adjusted, to further improve the safety and stability of unmanned aerial vehicle.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to a method and apparatus for optimizing UAV mission trajectories based on deep learning. Background Technology

[0002] Unmanned aerial vehicles (UAVs) are widely used in various fields due to their small size, low cost, and strong environmental adaptability. Current research on UAV trajectory planning often treats UAVs as point masses, primarily aiming to prevent collisions with known obstacles. This research requires an environmental acquisition module, including cameras, lidar, ultrasonic sensors, infrared sensors, map data interfaces, weather sensors, communication modules, and data fusion and processing modules, to detect the terrain, obstacle locations, weather conditions, and the positions of other aircraft, thereby avoiding collisions or enabling cooperative flight.

[0003] However, in practical applications, changes in environmental obstacles and weather can make it difficult to predict and adjust the original flight path in a timely manner. During operation, the drone may be damaged by obstacles that move onto the flight path, or the weather may change, causing the originally set path to deviate, resulting in the originally planned drone flight path no longer being smooth. Summary of the Invention

[0004] This invention provides a method and apparatus for optimizing UAV mission trajectories based on deep learning, thereby improving the accuracy and practicality of UAV mission trajectory optimization.

[0005] To address the aforementioned technical problems, this invention provides a deep learning-based method for optimizing UAV mission trajectories, comprising the following steps:

[0006] In response to the scheduling task execution signal, the corresponding scheduling task is retrieved and parsed to obtain task scheduling information, and environmental information of the UAV to be scheduled is collected and obtained based on the task scheduling information.

[0007] An initial task trajectory is generated based on the environmental information and the task scheduling information. The environmental information, the task scheduling information, and the environmental prediction information are used as model optimization parameters. The initial task trajectory is optimized using a deep learning model to obtain an optimized task trajectory. The environmental prediction information is obtained by predicting the surrounding environment of the initial task trajectory.

[0008] The optimized task trajectory is sent to the drone to be scheduled, so that the drone to be scheduled can execute the scheduling task according to the optimized task trajectory, and the task trajectory information is collected and fed back in real time.

[0009] The UAV mission trajectory optimization method provided by this invention retrieves and parses the corresponding scheduling task based on the response scheduling task execution signal, thereby obtaining task scheduling information to determine the UAV to execute the scheduling task, as well as the task's start and destination addresses. After obtaining the task scheduling information, the system then uses this information to obtain environmental information surrounding the UAV about to execute the scheduling task, thereby determining whether the UAV is currently suitable for executing the task and providing reference data for subsequent task execution trajectory planning.

[0010] Once the task information and the surrounding environment of the UAV are determined, the system can plan the initial mission trajectory for the UAV. The task information, environmental information, and environmental predictions obtained based on these are used as optimization parameters for the deep learning model. The deep learning model optimizes the initial mission trajectory. Based on the task and environmental information, the deep learning model learns through interaction with the environment to obtain the optimal operating trajectory for the UAV to perform the scheduled task under specific task and environmental constraints. The environmental predictions obtained based on the task and environmental information can predict the possible movement trajectories of various abnormal objects or weather conditions in the initial mission trajectory. This determines whether the predicted movement or weather conditions will cause inconvenience to the UAV's mission execution, and then incorporates these predictions into the deep learning model, allowing the model to learn and avoid these predicted situations. This avoids UAV collisions or cooperative flight, and also prevents foreign objects from moving onto the operating trajectory and causing damage to the UAV. Furthermore, the predicted environmental conditions can be used to adjust the UAV's flight speed and altitude, ensuring the safety of the UAV mission execution.

[0011] Once the optimized mission trajectory is determined, the system sends it to the drone to be scheduled, so that the drone can execute the scheduling mission according to the optimized trajectory. At the same time, during the mission execution, the system collects mission trajectory data in real time through sensors and feeds it back to the system. The system then makes subsequent trajectory adjustments based on the real-time data, further reducing the probability of the drone colliding with abnormal objects during mission execution, as well as the probability of trajectory deviation due to changes in weather conditions.

[0012] As a preferred example, the step of collecting and obtaining environmental information of the UAV to be scheduled based on the task scheduling information specifically includes:

[0013] Based on the task scheduling information, a number of corresponding address numbers are determined, and then a number of meteorological sensors, environmental acquisition components, UAV communication components, and terrain acquisition components are determined based on the number of address numbers.

[0014] The system acquires corresponding meteorological data, environmental data, obstacle data, and terrain data through a number of meteorological sensors, environmental acquisition components, UAV communication components, and terrain acquisition components, and integrates the meteorological data, environmental data, obstacle data, and terrain data into the environmental information.

[0015] To improve the accuracy and usability of the generated initial mission trajectory, when collecting environmental information around the UAV based on mission scheduling information, the system first determines the addresses involved in the scheduled mission, i.e., several address numbers, based on the determined address numbers. Then, it identifies the corresponding meteorological sensors, environmental acquisition components, UAV communication components, and terrain acquisition components present at each address, and acquires the data collected by each of these components. Therefore, the environmental information obtained by the system includes not only meteorological and obstacle data, but also terrain and environmental data. By integrating these various types of data, more comprehensive and accurate environmental information can be obtained, providing more reliable and comprehensive foundational data for subsequent trajectory optimization by deep learning models, and also providing data support for UAV decision-making and control.

[0016] As a preferred example, the step of generating the initial task trajectory based on the environmental information and the task scheduling information specifically includes:

[0017] The environmental information and the task scheduling information are integrated into initial trajectory information. The corresponding task trajectory database is retrieved according to the task scheduling information. Then, the initial trajectory information is used to match and filter in the task trajectory database, and the corresponding matching results are output.

[0018] If the matching result is that there is a matching degree between the corresponding task trajectory data and the initial trajectory information that is greater than or equal to a preset matching threshold, then the task trajectory data is integrated into the initial task trajectory and output.

[0019] If the matching result is that there is no matching degree between the task trajectory data and the initial trajectory information that is greater than or equal to the matching threshold, then the task trajectory data with the highest matching degree will be output as the initial task trajectory.

[0020] To further improve the accuracy, reliability, and usability of the generated initial task trajectory, the system generates the initial task trajectory by matching the initial trajectory information from the above two types of information with the task trajectory database retrieved based on the task scheduling information. Depending on the matching result, the system will execute different trajectory generation methods.

[0021] If the matching result indicates the existence of task trajectory data with a matching degree exceeding a preset threshold, it means that the drone corresponding to the matched task trajectory data not only performed a scheduling task highly similar to the scheduling task to be performed by the drone in this instance, but also that the environmental state of the drone performing the task was highly similar to the environmental state in this instance. Therefore, if only one task trajectory data exceeding the threshold is matched, this trajectory data will be output as the initial task trajectory; if multiple trajectory data with matching degrees exceeding the threshold are matched, the multiple matched trajectory data will be integrated to generate one as the initial task trajectory for output.

[0022] If no trajectory data exceeding the threshold is found, the trajectory data with the highest matching degree among all participating trajectory data is selected as the initial task trajectory for output. This matching method further improves the accuracy, reliability, and usability of the generated initial task trajectory, and also provides a good data foundation for subsequent system trajectory optimization.

[0023] As a preferred example, the step of using the environmental information, the task scheduling information, and the environmental prediction information as model optimization parameters, and optimizing the initial task trajectory using a deep learning model to obtain an optimized task trajectory, specifically involves:

[0024] The system tracks and collects environmental data of the initial task trajectory and motion trajectory data of abnormal objects in the trajectory in real time. The environmental data is then input into a preset random forest model for environmental prediction, and the corresponding environmental prediction data is output.

[0025] Simultaneously, the motion trajectory data is subjected to trajectory detection and motion prediction using preset computer vision technology, and corresponding motion trajectory prediction data is output. The environmental prediction data and the motion trajectory prediction data are integrated into the environmental prediction information.

[0026] The environmental prediction information, the environmental information, and the task scheduling information are used as model optimization parameters and input into the deep learning model. The initial task trajectory is also input into the deep learning model for model optimization and solution, and the optimized task trajectory is output.

[0027] After obtaining the initial task trajectory, the system will further optimize the initial task trajectory based on environmental information, task scheduling information, and environmental prediction information obtained from the prediction of the surrounding environment of the initial task trajectory. By strengthening the interaction with the environment through a deep learning model, the system can further optimize the initial task trajectory and improve the practicality and reliability of the optimized trajectory.

[0028] By real-time tracking and collection of environmental data and anomalous object trajectory data at various locations along the determined initial mission trajectory, and inputting the collected environmental data into a random forest model for prediction, environmental prediction data for the area surrounding the initial mission trajectory can be obtained. The anomalous object trajectory is detected and predicted using computer vision technology to obtain its predicted trajectory data for the future time period. These two types of prediction data, along with environmental information and mission scheduling information, are input as model optimization parameters into a deep learning model. The model then optimizes the initial mission trajectory based on these parameters, resulting in an optimized mission trajectory derived from the two types of prediction data, the UAV's location environment information, and the mission scheduling information.

[0029] As a preferred example, the real-time acquisition and feedback of task trajectory information specifically includes:

[0030] The task trajectory information is collected and fed back in real time. Based on the task environment information and abnormal motion information in the task trajectory information, environmental prediction and motion trajectory prediction are performed respectively to obtain task environment prediction information and task motion trajectory prediction information.

[0031] The deep learning model performs trajectory optimization processing on the optimized task trajectory based on the task environment prediction information and the task motion trajectory prediction information to obtain an adjusted task trajectory. The adjusted task trajectory is then sent to the drone to be scheduled so that the drone to be scheduled can adjust the execution path of the scheduled task according to the adjusted task trajectory.

[0032] Simultaneously, the task trajectory information is associated with the task scheduling information and the optimized task trajectory, and the associated data is stored as task trajectory data in the task trajectory database.

[0033] After receiving the mission trajectory information from the UAV, the system performs environmental prediction and trajectory prediction based on the mission environment information and abnormal object motion information in the information. At the same time, it also associates the mission trajectory information with the mission scheduling information and the optimized mission trajectory and stores it in the mission trajectory database, so that it can be called when the system generates the initial mission trajectory later.

[0034] For the predicted mission environment and mission trajectory information, the recurrent deep learning model performs secondary optimization on the optimized mission trajectory, outputting a corresponding adjusted mission trajectory. This adjusted trajectory is used to prevent collisions and damage to the drone due to abnormal objects or changes in weather or terrain along the optimized trajectory. Therefore, by collecting and feeding back mission trajectory information in real time from the drone, the optimized mission trajectory can be adjusted in real time, improving the accuracy and usability of the adjusted trajectory, and ultimately enhancing the efficiency and safety of drone mission execution.

[0035] Accordingly, this invention also provides a deep learning-based UAV mission trajectory optimization device, which includes a mission information acquisition module, a mission trajectory optimization module, and a mission trajectory feedback module.

[0036] The task information acquisition module is used to respond to the scheduling task execution signal, retrieve and parse the corresponding scheduling task, obtain task scheduling information, and collect and acquire environmental information of the UAV to be scheduled based on the task scheduling information.

[0037] The task trajectory optimization module is used to generate an initial task trajectory based on the environmental information and the task scheduling information, and to use the environmental information, the task scheduling information and the environmental prediction information as model optimization parameters to optimize the initial task trajectory through a deep learning model to obtain an optimized task trajectory.

[0038] The task trajectory feedback module is used to send the optimized task trajectory to the drone to be scheduled, so that the drone to be scheduled can execute the scheduling task according to the optimized task trajectory, and collect and feedback task trajectory information in real time.

[0039] As a preferred example, the task information acquisition module collects and obtains environmental information of the UAV to be scheduled based on the task scheduling information, specifically including:

[0040] Based on the task scheduling information, a number of corresponding address numbers are determined, and then a number of meteorological sensors, environmental acquisition components, UAV communication components, and terrain acquisition components are determined based on the number of address numbers.

[0041] The system acquires corresponding meteorological data, environmental data, obstacle data, and terrain data through a number of meteorological sensors, environmental acquisition components, UAV communication components, and terrain acquisition components, and integrates the meteorological data, environmental data, obstacle data, and terrain data into the environmental information.

[0042] As a preferred example, the task trajectory optimization module generates an initial task trajectory based on the environmental information and the task scheduling information, specifically as follows:

[0043] The environmental information and the task scheduling information are integrated into initial trajectory information. The corresponding task trajectory database is retrieved according to the task scheduling information. Then, the initial trajectory information is used to match and filter in the task trajectory database, and the corresponding matching results are output.

[0044] If the matching result is that there is a matching degree between the corresponding task trajectory data and the initial trajectory information that is greater than or equal to a preset matching threshold, then the task trajectory data is integrated into the initial task trajectory and output.

[0045] If the matching result is that there is no matching degree between the task trajectory data and the initial trajectory information that is greater than or equal to the matching threshold, then the task trajectory data with the highest matching degree will be output as the initial task trajectory.

[0046] As a preferred example, the task trajectory optimization module uses the environmental information, the task scheduling information, and the environmental prediction information as model optimization parameters, and optimizes the initial task trajectory using a deep learning model to obtain an optimized task trajectory, specifically as follows:

[0047] The system tracks and collects environmental data of the initial task trajectory and the trajectory data of abnormal objects in the trajectory in real time. The environmental data is then input into a preset random forest model for environmental prediction, and the corresponding environmental prediction data is output.

[0048] Simultaneously, the motion trajectory data is subjected to trajectory detection and motion prediction using preset computer vision technology, and corresponding motion trajectory prediction data is output. The environmental prediction data and the motion trajectory prediction data are integrated into the environmental prediction information.

[0049] The environmental prediction information, the environmental information, and the task scheduling information are used as model optimization parameters and input into the deep learning model. The initial task trajectory is also input into the deep learning model for model optimization and solution, and the optimized task trajectory is output.

[0050] As a preferred example, the task trajectory feedback module collects and feeds back task trajectory information in real time, specifically including:

[0051] The task trajectory information is collected and fed back in real time. Based on the task environment information and abnormal motion information in the task trajectory information, environmental prediction and motion trajectory prediction are performed respectively to obtain task environment prediction information and task motion trajectory prediction information.

[0052] The deep learning model performs trajectory optimization processing on the optimized task trajectory based on the task environment prediction information and the task motion trajectory prediction information to obtain an adjusted task trajectory. The adjusted task trajectory is then sent to the drone to be scheduled so that the drone to be scheduled can adjust the execution path of the scheduled task according to the adjusted task trajectory.

[0053] Simultaneously, the task trajectory information is associated with the task scheduling information and the optimized task trajectory, and the associated data is stored as task trajectory data in the task trajectory database. Attached Figure Description

[0054] Figure 1 : A flowchart illustrating an embodiment of the deep learning-based UAV mission trajectory optimization method provided by the present invention;

[0055] Figure 2 : A schematic diagram of the structure of an embodiment of the deep learning-based UAV mission trajectory optimization device provided by the present invention;

[0056] Figure 3 : A schematic diagram of another embodiment of the deep learning-based UAV mission trajectory optimization device provided by the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Example 1

[0059] Please refer to Figure 1 This is a flowchart illustrating an embodiment of the deep learning-based UAV mission trajectory optimization method provided by the present invention, including steps 101 to 103, each step being as follows:

[0060] Step 101: In response to the scheduling task execution signal, retrieve and parse the corresponding scheduling task to obtain task scheduling information, and collect and acquire environmental information of the UAV to be scheduled based on the task scheduling information.

[0061] The UAV mission trajectory optimization method provided in this embodiment of the invention retrieves and parses the corresponding scheduling task based on the response scheduling task execution signal, thereby obtaining task scheduling information to determine the UAV to execute the scheduling task, as well as the task's start address and destination address. After obtaining the task scheduling information, the system obtains the environmental information around the UAV about to execute the scheduling task based on the task scheduling information, thereby determining whether the UAV to be scheduled is suitable for executing the task at this time, and providing reference data for subsequent task execution trajectory planning.

[0062] In this embodiment, the step of collecting and obtaining environmental information of the UAV to be scheduled based on the task scheduling information specifically includes:

[0063] Based on the task scheduling information, a number of corresponding address numbers are determined, and then a number of meteorological sensors, environmental acquisition components, UAV communication components, and terrain acquisition components are determined based on the number of address numbers.

[0064] The system acquires corresponding meteorological data, environmental data, obstacle data, and terrain data through a number of meteorological sensors, environmental acquisition components, UAV communication components, and terrain acquisition components, and integrates the meteorological data, environmental data, obstacle data, and terrain data into the environmental information.

[0065] To improve the accuracy and usability of the generated initial mission trajectory, when collecting environmental information around the UAV based on mission scheduling information, the system first determines the addresses involved in the scheduled mission, i.e., several address numbers, based on the determined address numbers. Then, it identifies the corresponding meteorological sensors, environmental acquisition components, UAV communication components, and terrain acquisition components present at each address, and acquires the data collected by each of these components. Therefore, the environmental information obtained by the system includes not only meteorological and obstacle data, but also terrain and environmental data. By integrating these various types of data, more comprehensive and accurate environmental information can be obtained, providing more reliable and comprehensive foundational data for subsequent trajectory optimization by deep learning models, and also providing data support for UAV decision-making and control.

[0066] In this embodiment, the system obtains data from corresponding meteorological sensors, environmental acquisition components, UAV communication components, and terrain acquisition components through several address numbers. Specifically, this includes: obtaining meteorological conditions at a given address, such as temperature, humidity, and wind speed, through meteorological sensors, which helps the UAV consider meteorological factors during flight; collecting relevant environmental data through environmental acquisition components, including cameras, lidar, ultrasonic sensors, and infrared sensors; obtaining the location information of other aircraft and air traffic control information through the UAV communication component, ensuring the flight safety of the UAV; and obtaining map information, including terrain, buildings, and roads, through the terrain acquisition component, i.e., the map data interface, to help the UAV plan its flight path and avoid obstacles.

[0067] Step 102: Generate an initial task trajectory based on the environmental information and the task scheduling information. Use the environmental information, the task scheduling information, and the environmental prediction information as model optimization parameters, and optimize the initial task trajectory using a deep learning model to obtain an optimized task trajectory. The environmental prediction information is obtained by predicting the surrounding environment of the initial task trajectory.

[0068] Once the task information and the surrounding environment of the UAV are determined, the system can plan the initial mission trajectory for the UAV. The task information, environmental information, and environmental predictions obtained based on these are used as optimization parameters for the deep learning model. The deep learning model optimizes the initial mission trajectory. Based on the task and environmental information, the deep learning model learns through interaction with the environment to obtain the optimal operating trajectory for the UAV to perform the scheduled task under specific task and environmental constraints. The environmental predictions obtained based on the task and environmental information can predict the possible movement trajectories of various abnormal objects or weather conditions in the initial mission trajectory. This determines whether the predicted movement or weather conditions will cause inconvenience to the UAV's mission execution, and then incorporates these predictions into the deep learning model, allowing the model to learn and avoid these predicted situations. This avoids UAV collisions or cooperative flight, and also prevents foreign objects from moving onto the operating trajectory and causing damage to the UAV. Furthermore, the predicted environmental conditions can be used to adjust the UAV's flight speed and altitude, ensuring the safety of the UAV mission execution.

[0069] Specifically, in this embodiment, generating the initial task trajectory based on the environmental information and the task scheduling information involves the following steps:

[0070] The environmental information and the task scheduling information are integrated into initial trajectory information. The corresponding task trajectory database is retrieved according to the task scheduling information. Then, the initial trajectory information is used to match and filter in the task trajectory database, and the corresponding matching results are output.

[0071] If the matching result is that there is a matching degree between the corresponding task trajectory data and the initial trajectory information that is greater than or equal to a preset matching threshold, then the task trajectory data is integrated into the initial task trajectory and output.

[0072] If the matching result is that there is no matching degree between the task trajectory data and the initial trajectory information that is greater than or equal to the matching threshold, then the task trajectory data with the highest matching degree will be output as the initial task trajectory.

[0073] To further improve the accuracy, reliability, and usability of the generated initial task trajectory, the system generates the initial task trajectory by matching the initial trajectory information from the above two types of information with the task trajectory database retrieved based on the task scheduling information. Depending on the matching result, the system will execute different trajectory generation methods.

[0074] If the matching result indicates the existence of task trajectory data with a matching degree exceeding a preset threshold, it means that the drone corresponding to the matched task trajectory data not only performed a scheduling task highly similar to the scheduling task to be performed by the drone in this instance, but also that the environmental state of the drone performing the task was highly similar to the environmental state in this instance. Therefore, if only one task trajectory data exceeding the threshold is matched, this trajectory data will be output as the initial task trajectory; if multiple trajectory data with matching degrees exceeding the threshold are matched, the multiple matched trajectory data will be integrated to generate one as the initial task trajectory for output.

[0075] If no trajectory data exceeding the threshold is found, the trajectory data with the highest matching degree among all participating trajectory data is selected as the initial task trajectory for output. This matching method further improves the accuracy, reliability, and usability of the generated initial task trajectory, and also provides a good data foundation for subsequent system trajectory optimization.

[0076] Furthermore, in this embodiment, the environmental information, the task scheduling information, and the environmental prediction information are used as model optimization parameters. A deep learning model is used to optimize the initial task trajectory to obtain an optimized task trajectory. Specifically:

[0077] The system tracks and collects environmental data of the initial task trajectory and motion trajectory data of abnormal objects in the trajectory in real time. The environmental data is then input into a preset random forest model for environmental prediction, and the corresponding environmental prediction data is output.

[0078] Simultaneously, the motion trajectory data is subjected to trajectory detection and motion prediction using preset computer vision technology, and corresponding motion trajectory prediction data is output. The environmental prediction data and the motion trajectory prediction data are integrated into the environmental prediction information.

[0079] The environmental prediction information, the environmental information, and the task scheduling information are used as model optimization parameters and input into the deep learning model. The initial task trajectory is also input into the deep learning model for model optimization and solution, and the optimized task trajectory is output.

[0080] After obtaining the initial task trajectory, the system will further optimize the initial task trajectory based on environmental information, task scheduling information, and environmental prediction information obtained from the prediction of the surrounding environment of the initial task trajectory. By strengthening the interaction with the environment through a deep learning model, the system can further optimize the initial task trajectory and improve the practicality and reliability of the optimized trajectory.

[0081] By real-time tracking and collection of environmental data and anomalous object trajectory data at various locations along the determined initial mission trajectory, and inputting the collected environmental data into a random forest model for prediction, environmental prediction data for the area surrounding the initial mission trajectory can be obtained. The anomalous object trajectory is detected and predicted using computer vision technology to obtain its predicted trajectory data for the future time period. These two types of prediction data, along with environmental information and mission scheduling information, are input as model optimization parameters into a deep learning model. The model then optimizes the initial mission trajectory based on these parameters, resulting in an optimized mission trajectory derived from the two types of prediction data, the UAV's location environment information, and the mission scheduling information.

[0082] Step 103: Send the optimized task trajectory to the drone to be scheduled, so that the drone to be scheduled can execute the scheduling task according to the optimized task trajectory, and collect and feedback task trajectory information in real time.

[0083] Once the optimized mission trajectory is determined, the system sends it to the drone to be scheduled, so that the drone can execute the scheduling mission according to the optimized trajectory. At the same time, during the mission execution, the system collects mission trajectory data in real time through sensors and feeds it back to the system. The system then makes subsequent trajectory adjustments based on the real-time data, further reducing the probability of the drone colliding with abnormal objects during mission execution, as well as the probability of trajectory deviation due to changes in weather conditions.

[0084] Specifically, the real-time acquisition and feedback of task trajectory information described in this embodiment includes:

[0085] The task trajectory information is collected and fed back in real time. Based on the task environment information and abnormal motion information in the task trajectory information, environmental prediction and motion trajectory prediction are performed respectively to obtain task environment prediction information and task motion trajectory prediction information.

[0086] The deep learning model performs trajectory optimization processing on the optimized task trajectory based on the task environment prediction information and the task motion trajectory prediction information to obtain an adjusted task trajectory. The adjusted task trajectory is then sent to the drone to be scheduled so that the drone to be scheduled can adjust the execution path of the scheduled task according to the adjusted task trajectory.

[0087] Simultaneously, the task trajectory information is associated with the task scheduling information and the optimized task trajectory, and the associated data is stored as task trajectory data in the task trajectory database.

[0088] After receiving the mission trajectory information from the UAV, the system performs environmental prediction and trajectory prediction based on the mission environment information and abnormal object motion information in the information. At the same time, it also associates the mission trajectory information with the mission scheduling information and the optimized mission trajectory and stores it in the mission trajectory database, so that it can be called when the system generates the initial mission trajectory later.

[0089] For the predicted mission environment and mission trajectory information, the recurrent deep learning model performs secondary optimization on the optimized mission trajectory, outputting a corresponding adjusted mission trajectory. This adjusted trajectory is used to prevent collisions and damage to the drone due to abnormal objects or changes in weather or terrain along the optimized trajectory. Therefore, by collecting and feeding back mission trajectory information in real time from the drone, the optimized mission trajectory can be adjusted in real time, improving the accuracy and usability of the adjusted trajectory, and ultimately enhancing the efficiency and safety of drone mission execution.

[0090] Example 2

[0091] See Figure 3 , Figure 3 This is a schematic diagram of another embodiment of the deep learning-based UAV mission trajectory optimization device provided by the present invention. Figure 3 As shown, the UAV mission trajectory optimization device includes an environment acquisition module 301, a mission planning module 302, a trajectory optimization module 303, an environment prediction module 304, and a recording and learning module 305.

[0092] Accordingly, based on the above-mentioned UAV trajectory optimization device, this embodiment of the invention also provides corresponding UAV trajectory optimization steps, the specific steps of which are as follows:

[0093] Step 1: Environment Acquisition. The environment acquisition module 301 is a device or system for acquiring information about the environment surrounding the UAV, collecting various data required for the UAV's flight, including terrain, obstacle locations, weather conditions, and the positions of other aircraft. Specifically, the environment acquisition module 301 is a device or system for acquiring information about the environment surrounding the UAV, collecting various data required for the UAV's flight, including terrain, obstacle locations, weather conditions, and the positions of other aircraft.

[0094] Step Two: Mission Planning. The mission planning module 302 plans the UAV's flight path based on mission requirements and environmental analysis results. Specific details include obstacle avoidance, optimizing flight efficiency, and considering battery life. Specifically, the mission planning module 302 is used to plan a path from the starting point to the target point based on mission requirements, and to confirm whether there are obstacles along the path and whether weather conditions will affect the UAV's movement based on environmental information.

[0095] Step 3: Trajectory optimization. The trajectory optimization module 303 combines the policy gradient method and Q-learning with DDPG to calculate and plan the flight path. Specifically, the trajectory optimization module 303 uses deep reinforcement learning (DRL) to optimize the UAV's flight trajectory. The DRL algorithm learns through interaction with the environment to find the optimal flight trajectory under specific task and environmental constraints.

[0096] Step 4: Environmental Prediction. The environmental prediction module 304 uses the monitored environmental weather and obstacles to determine if they exhibit movement trajectories. The algorithm is Random Forest, an ensemble learning method used for classification and regression tasks. Specifically, the environmental prediction module 304 uses computer vision technology to detect and track objects in the environment, such as obstacles or other aircraft. By monitoring the position, speed, and trajectory of objects in real time, it predicts their future behavior, thereby avoiding collisions or enabling cooperative flight, and preventing foreign objects from moving onto the drone's trajectory and causing damage.

[0097] Step 5: Actual flight. Calculate the optimal flight trajectory based on the mission planning, trajectory planning, and environmental prediction, and send the determined optimal flight trajectory to the UAV so that the UAV can operate according to the flight trajectory.

[0098] Step Six: Record Learning. The recording learning module 305 records real-time data from the flight trajectory. Based on specific flight time, flight route, flight altitude, speed, battery level, etc., the flight plan is optimized. The optimized flight plan is then used to repeatedly execute the task, ensuring that each flight achieves the expected results. Specifically, the recording learning module 305 uses the drone's flight recorder or flight controller to record detailed data for each flight, including flight time, flight route, flight altitude, speed, battery level, etc.

[0099] Step 7: Plan Adjustment. Based on real-time monitoring of obstacles or other obstructions, replan the flight path and adjust the flight speed and altitude according to changes in weather conditions to ensure safety.

[0100] Compared to the system in Implementation 1 that matches and outputs the corresponding initial task trajectory in the task trajectory database based on environmental information and task scheduling information, Implementation 2 directly generates the corresponding initial task trajectory based on task scheduling information and then adjusts it based on environmental information.

[0101] Specifically, as described in step three above, trajectory optimization is performed by combining the policy gradient method and Q-learning with DDPG. The specific process is as follows:

[0102] DDPG is a policy gradient-based method applicable to continuous action spaces. It combines policy gradient methods and Q-learning, and its formula is: Policy Network:

[0103]

[0104] in, It is the output of the policy network. These are the network parameters, and π(a|s) is the policy. The formula for the Q-network is as follows:

[0105] Q(s,a;θ)≈Q μ (s,a)

[0106] Where Q(s, a; θ) is the output of the network, Q μ (s, a) is the Q-value under policy μ. The corresponding measurement gradient is as follows:

[0107]

[0108] Where J is the performance index, ρ u This refers to the state distribution under the policy. Using the above formula, a trajectory optimization component based on DDPG, combining the policy gradient method and Q-learning, can be constructed to optimize the trajectory and find the optimal flight trajectory under specific task and environmental constraints.

[0109] Simultaneously, the environmental prediction module 304 in step four specifically performs environmental modeling based on the environmental prediction, using a meteorological model to predict weather changes over a future period, such as wind speed, wind direction, temperature, and humidity, to construct a three-dimensional terrain model, predicting terrain features and obstacle distribution along the UAV's flight path, and using this as reference data to generate the optimal mission execution trajectory. Furthermore, the calculation formula for the random forest ensemble learning method is as follows:

[0110]

[0111] in, Here, B is the predicted value, and f is the number of trees. b (x) is the prediction of the b-th tree. The specific formula for the Support Vector Machine (SVM), a supervised learning model used for classification and regression, is as follows:

[0112]

[0113] Where, α i It is a Lagrange multiplier, α i It is a category label, K(X, X). i The kernel function () is composed of multiple decision trees. These decision trees are constructed independently during training, and the random forest improves the overall prediction performance and generalization ability by combining the prediction results of these decision trees.

[0114] Step seven, which involves replanning the flight path based on real-time monitored obstacles or other obstructions, specifically involves reallocating UAV resources, such as sensor usage and battery power, according to mission requirements. The algorithm is based on real-time data streams, and the specific time-driven sliding window algorithm is shown below:

[0115] W t ={d t-T+1 d t-T+2 ,....,d t}

[0116] Among them, W t It is the window of time t, d i Here, T represents the data points in the data stream, and T is the size of the window. A data-driven sliding window works as follows:

[0117] W n ={d n-k+1 d n-k+2 ,....,d n}

[0118] Among them, W nThis is a window containing k data points. After determining the sliding window, the system will process the real-time data using a micro-batch processing model. Each batch contains a certain number of data points, and the specific calculation formula for a fixed-size micro-batch is as follows:

[0119] B = {d1, d2, ..., d} m}

[0120] Where B is a batch containing m data points, and the dynamically sized micro-batch is:

[0121] B = {d1, d2, ..., d} n}

[0122] Here, B represents a batch containing n data points, and n can change dynamically according to the characteristics of the data volume.

[0123] To better illustrate the working principle and steps of the deep learning-based UAV mission trajectory optimization method and device of the present invention, please refer to the relevant description above, but not limited to.

[0124] Accordingly, see Figure 2 , Figure 2 This is a schematic diagram of one embodiment of the deep learning-based UAV mission trajectory optimization device provided by the present invention. Figure 2 As shown, the UAV mission trajectory optimization device includes a mission information acquisition module 201, a mission trajectory optimization module 202, and a mission trajectory feedback module 203.

[0125] The task information acquisition module 201 is used to respond to the scheduling task execution signal, retrieve and parse the corresponding scheduling task, obtain task scheduling information, and collect and acquire environmental information of the UAV to be scheduled based on the task scheduling information.

[0126] Furthermore, the task information acquisition module 201 acquires and obtains the environmental information of the UAV to be scheduled based on the task scheduling information, specifically including:

[0127] Based on the task scheduling information, a number of corresponding address numbers are determined, and then a number of meteorological sensors, environmental acquisition components, UAV communication components, and terrain acquisition components are determined based on the number of address numbers. Corresponding meteorological data, environmental data, obstacle data, and terrain data are obtained through the determined number of meteorological sensors, environmental acquisition components, UAV communication components, and terrain acquisition components, and the meteorological data, environmental data, obstacle data, and terrain data are integrated into the environmental information.

[0128] The task trajectory optimization module 202 is used to generate an initial task trajectory based on the environmental information and the task scheduling information, and use the environmental information, the task scheduling information and the environmental prediction information as model optimization parameters to optimize the initial task trajectory through a deep learning model to obtain an optimized task trajectory.

[0129] Furthermore, the task trajectory optimization module 202 generates an initial task trajectory based on the environmental information and the task scheduling information, specifically as follows:

[0130] The environmental information and the task scheduling information are integrated into initial trajectory information. The corresponding task trajectory database is retrieved according to the task scheduling information. Then, the initial trajectory information is used to match and filter in the task trajectory database, and the corresponding matching results are output.

[0131] If the matching result indicates that there is a matching degree between the corresponding task trajectory data and the initial trajectory information that is greater than or equal to a preset matching threshold, then the task trajectory data is integrated into the initial task trajectory and output; if the matching result indicates that there is no matching degree between the task trajectory data and the initial trajectory information that is greater than or equal to the matching threshold, then the task trajectory data with the highest matching degree is output as the initial task trajectory.

[0132] Furthermore, the task trajectory optimization module 202 uses the environmental information, the task scheduling information, and the environmental prediction information as model optimization parameters, and optimizes the initial task trajectory using a deep learning model to obtain an optimized task trajectory, specifically as follows:

[0133] The system tracks and collects environmental data of the initial task trajectory and the trajectory data of abnormal objects in the trajectory in real time. The environmental data is then input into a preset random forest model for environmental prediction, and the corresponding environmental prediction data is output.

[0134] Simultaneously, the motion trajectory data is subjected to trajectory detection and motion prediction using preset computer vision technology, and corresponding motion trajectory prediction data is output. The environmental prediction data and the motion trajectory prediction data are integrated into the environmental prediction information. The environmental prediction information, the environmental information, and the task scheduling information are used as model optimization parameters and input into the deep learning model. The initial task trajectory is input into the deep learning model for model optimization and solution, and the optimized task trajectory is output.

[0135] The task trajectory feedback module 203 is used to send the optimized task trajectory to the drone to be scheduled, so that the drone to be scheduled can execute the scheduling task according to the optimized task trajectory, and collect and feedback task trajectory information in real time.

[0136] Furthermore, the task trajectory feedback module 203 collects and provides feedback on task trajectory information in real time, specifically including:

[0137] The system collects and feeds back the task trajectory information in real time. Based on the task environment information and abnormal motion information within the task trajectory information, it performs environment prediction and motion trajectory prediction to obtain task environment prediction information and task motion trajectory prediction information. Then, using the deep learning model, it performs trajectory optimization processing on the optimized task trajectory based on the task environment prediction information and task motion trajectory prediction information to obtain an adjusted task trajectory. The adjusted task trajectory is then sent to the drone to be scheduled, so that the drone to be scheduled can adjust the execution path of the scheduled task according to the adjusted task trajectory.

[0138] Simultaneously, the task trajectory information is associated with the task scheduling information and the optimized task trajectory, and the associated data is stored as task trajectory data in the task trajectory database.

[0139] In summary, this invention provides a method and apparatus for optimizing UAV mission trajectories based on deep learning. The method retrieves and parses the scheduled task based on the response task execution signal to obtain task scheduling information. It then collects environmental information of the UAV to be scheduled based on the task scheduling information. An initial mission trajectory is generated based on the environmental information and the task scheduling information. The initial mission trajectory is optimized using the above two types of information, along with environmental prediction information obtained from environmental information prediction, as optimization parameters of the deep learning model. The optimized mission trajectory is then sent to the UAV, enabling the UAV to execute the scheduled task according to the optimized trajectory and provide feedback on the mission trajectory information. By learning through interaction with the environment, the optimal operating trajectory of the UAV is obtained. Adjustments to the UAV's flight speed and altitude are made using environmental prediction data, ensuring the safety of UAV mission execution. Furthermore, trajectory adjustments based on real-time collected data further improve the safety and stability of the UAV.

[0140] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1.A method for optimizing a task trajectory of a UAV based on deep learning, characterized in that, The method comprises the following steps: in response to a scheduling task execution signal, calling and analyzing a corresponding scheduling task to obtain task scheduling information, collecting and obtaining environment information of a to-be-scheduled unmanned aerial vehicle according to the task scheduling information; generating an initial task trajectory according to the environment information and the task scheduling information, taking the environment information, the task scheduling information and environment prediction information as model optimization parameters, optimizing the initial task trajectory through a deep learning model to obtain an optimized task trajectory; wherein the environment prediction information is obtained by predicting the surrounding environment of the initial task trajectory; sending the optimized task trajectory to the to-be-scheduled unmanned aerial vehicle, so that the to-be-scheduled unmanned aerial vehicle executes the scheduling task according to the optimized task trajectory and collects feedback task trajectory information in real time; the collecting and obtaining of the environment information of the to-be-scheduled unmanned aerial vehicle according to the task scheduling information specifically comprises: determining a plurality of address numbers corresponding to the task scheduling information, and then determining a plurality of weather sensors, a plurality of environment collection components, unmanned aerial vehicle communication components and terrain collection components according to the plurality of address numbers; obtaining corresponding weather data, environment data, obstacle data and terrain data through the determined plurality of weather sensors, plurality of environment collection components, unmanned aerial vehicle communication components and terrain collection components, and integrating the weather data, environment data, obstacle data and terrain data into the environment information; the generating of the initial task trajectory according to the environment information and the task scheduling information specifically comprises: integrating the environment information and the task scheduling information into initial trajectory information, calling a corresponding task trajectory database according to the task scheduling information, and then performing matching and screening in the task trajectory database according to the initial trajectory information to output a corresponding matching result; if the matching result is that the matching degree of the initial trajectory information and corresponding task trajectory data is greater than or equal to a preset matching threshold, integrating the task trajectory data into the initial task trajectory for output; if the matching result is that there is no matching degree of the initial trajectory information and the task trajectory data greater than or equal to the matching threshold, outputting the task trajectory data with the highest matching degree as the initial task trajectory. 2.The method of claim 1, wherein, the taking of the environment information, the task scheduling information and the environment prediction information as model optimization parameters, the optimization of the initial task trajectory through a deep learning model to obtain an optimized task trajectory, specifically comprises: tracking and collecting environment data of the initial task trajectory and motion trajectory data of an abnormal object in the trajectory in real time, inputting the environment data into a preset random forest model for environment prediction to output corresponding environment prediction data; at the same time, detecting and predicting the motion trajectory data through a preset computer vision technology to output corresponding motion trajectory prediction data, and integrating the environment prediction data and the motion trajectory prediction data into the environment prediction information; The environment prediction information, the environment information and the task scheduling information are input into the deep learning model as the model optimization parameters, and the initial task trajectory is input into the deep learning model for model optimization solving, and the optimized task trajectory is output. 3.The method of claim 1, wherein, The real-time collection and feedback of the task trajectory information are specifically as follows: The task environment information and the task abnormal motion information in the task trajectory information are used for environment prediction and motion trajectory prediction, respectively, to obtain task environment prediction information and task motion trajectory prediction information; The deep learning model is used to perform trajectory optimization processing on the optimized task trajectory according to the task environment prediction information and the task motion trajectory prediction information, to obtain an adjusted task trajectory, which is then sent to the unmanned aerial vehicle to be scheduled, so that the unmanned aerial vehicle to be scheduled adjusts the execution path of the scheduling task according to the adjusted task trajectory; Meanwhile, the task trajectory information is associated with the task scheduling information and the optimized task trajectory, and the associated data is stored as task trajectory data in a task trajectory database. 4.A deep learning based unmanned aerial vehicle task trajectory optimization apparatus, characterized in that, The unmanned aerial vehicle task trajectory optimization device comprises a task information collection module, a task trajectory optimization model and a task trajectory feedback module. The task information collection module is configured to, in response to a scheduling task execution signal, call and analyze a corresponding scheduling task to obtain task scheduling information, and collect and obtain environment information of an unmanned aerial vehicle to be scheduled according to the task scheduling information. The task trajectory optimization model is configured to generate an initial task trajectory according to the environment information and the task scheduling information, and use the environment information, the task scheduling information and environment prediction information as model optimization parameters to perform trajectory optimization on the initial task trajectory by a deep learning model to obtain an optimized task trajectory. The task trajectory feedback module is configured to send the optimized task trajectory to the unmanned aerial vehicle to be scheduled, so that the unmanned aerial vehicle to be scheduled executes the scheduling task according to the optimized task trajectory, and real-time collection and feedback of task trajectory information are performed. The task information collection module collects and obtains environment information of an unmanned aerial vehicle to be scheduled according to the task scheduling information, specifically as follows: A plurality of address numbers are determined according to the task scheduling information, and a plurality of meteorological sensors, a plurality of environment collection components, an unmanned aerial vehicle communication component and a terrain collection component are determined according to the plurality of address numbers. Corresponding meteorological data, environment data, obstacle data and terrain data are obtained by the plurality of meteorological sensors, the plurality of environment collection components, the unmanned aerial vehicle communication component and the terrain collection component, respectively, and the meteorological data, the environment data, the obstacle data and the terrain data are integrated into the environment information. The task trajectory optimization model generates an initial task trajectory according to the environment information and the task scheduling information, specifically as follows: Integrate the environment information and the task scheduling information into initial trajectory information, and call a corresponding task trajectory database according to the task scheduling information, and then perform matching and screening in the task trajectory database according to the initial trajectory information, and output a corresponding matching result; If the matching result is that the matching degree of the corresponding task trajectory data and the initial trajectory information is greater than or equal to a preset matching threshold, the task trajectory data is integrated into the initial task trajectory and output; If the matching result is that there is no matching degree of the task trajectory data and the initial trajectory information greater than or equal to the matching threshold, the task trajectory data with the highest matching degree is output as the initial task trajectory. 5.The unmanned aerial vehicle task trajectory optimization apparatus based on deep learning of claim 4, wherein, The task trajectory optimization module takes the environment information, the task scheduling information and environment prediction information as model optimization parameters, and optimizes the initial task trajectory through a deep learning model to obtain an optimized task trajectory, specifically: Real-time tracking and collecting environment data of the initial task trajectory and motion trajectory data of abnormal objects in the trajectory, inputting the environment data into a preset random forest model for environment prediction, and outputting corresponding environment prediction data; At the same time, the motion trajectory data is detected and motion predicted through a preset computer vision technology, and corresponding motion trajectory prediction data is output, and the environment prediction data and the motion trajectory prediction data are integrated into the environment prediction information; The environment prediction information, the environment information and the task scheduling information are input into the deep learning model as the model optimization parameters, and the initial task trajectory is input into the deep learning model for model optimization solution, and the optimized task trajectory is output. 6.The unmanned aerial vehicle task trajectory optimization apparatus based on deep learning of claim 4, wherein, The task trajectory feedback module collects feedback task trajectory information in real time, specifically including: Real-time collection and feedback of the task trajectory information, environment prediction and motion trajectory prediction according to task environment information and task abnormal motion information in the task trajectory information, respectively, to obtain task environment prediction information and task motion trajectory prediction information; The deep learning model is used to optimize the optimized task trajectory according to the task environment prediction information and the task motion trajectory prediction information, to obtain an adjusted task trajectory, and then the adjusted task trajectory is sent to the unmanned aerial vehicle to be scheduled, so that the unmanned aerial vehicle to be scheduled adjusts the execution path of the scheduling task according to the adjusted task trajectory; At the same time, the task trajectory information is associated with the task scheduling information and the optimized task trajectory, and the associated data is stored as task trajectory data in a task trajectory database.

Images