Multi-Drone Network Operation Plan via Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Formulating an effective operation plan for a multi-drone network that performs data sensing and communication relay tasks is challenging due to mobility, communication distance restrictions, and the need for experienced personnel, making it difficult to maximize cooperative synergy and optimize task execution.
Innovation Solution
A method using reinforcement learning, specifically the multi-agent deep deterministic policy gradient (MADDPG) algorithm, to automatically generate a cooperative operation plan for multiple drones, defining observation, action, and reward models to optimize data sensing and communication relay tasks in real-time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual operation plan formulation by experienced personnel is used, then task execution quality is improved, but time consumption and operational burden increase significantly
Solution Approach 1:
The system enables autonomous self-planning through reinforcement learning algorithms. The multi-agent system learns to generate operation plans independently without human intervention, allowing the drone network to serve itself in plan formulation while maintaining high execution quality through learned optimization strategies
Solution Approach 2:
Manual mechanical planning processes are replaced with intelligent algorithms. The reinforcement learning-based automated planning system substitutes the human expert's mechanical decision-making process with computational models that learn optimal plans through training, dramatically reducing time consumption while maintaining reliability
2Reliability
If traditional operation planning methods are used, then communication distance restrictions can be managed, but cooperative synergy between multiple drones cannot be maximized
Solution Approach 1:
The system employs dynamic plan generation that adapts to real-time communication conditions. The reinforcement learning agents learn to dynamically adjust their operations based on current communication link states, enabling the network to maximize cooperative synergy while maintaining stability under distance restrictions through flexible, adaptive decision-making
Solution Approach 2:
The multi-agent reinforcement learning framework provides a universal planning mechanism that simultaneously handles multiple objectives including communication maintenance, task coordination, and synergy optimization. This unified approach allows the system to manage communication constraints while maximizing overall network productivity through integrated decision-making
3Ease of operation
If automated plan generation is implemented, then operational burden is reduced, but handling strict communication restrictions and high mobility becomes more difficult
Solution Approach 1:
The system performs preliminary learning and adaptation during a training phase before actual operation. The reinforcement learning agents are pre-trained on various communication scenarios and mobility patterns, allowing them to handle complex restrictions automatically during deployment without requiring real-time human intervention or complex manual configuration
Solution Approach 2:
The reinforcement learning algorithm acts as an intermediary between the complex communication constraints and the automated planning process. It translates strict communication restrictions and mobility requirements into learnable parameters and rewards, enabling the automated system to handle complexity internally while presenting a simple interface for operation
Data Source
AI summary
The present disclosure relates to a method and apparatus for generating a multi-drone network operation plan based on reinforcement learning. The method of generating a multi-drone network operation plan based on reinforcement learning includes defining a reinforcement learning hyperparameter and training an actor neural network for each drone agent by using a multi-agent deep deterministic policy gradient (MADDPG) algorithm based on the defined hyperparameter, generating Markov game formalization information based on multi-drone network task information and generating state-action history information by using the trained actor neural network based on the formalization information, and generating a multi-drone network operation plan based on the state-action history information.