Multi-UAV Continuous Movement Control for Energy-Efficient Coverage

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing UAV movement control methods using Q-learning result in imprecise control due to discrete mobile policies, leading to suboptimal communication coverage and energy inefficiency.

Innovation Solution

A multi-UAV continuous movement control method employing a deep deterministic policy gradient (DDPG) model that determines control information based on observation data, including energy consumption and coverage fairness, to optimize UAV movement for energy-efficient communication coverage.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If Q-learning is used to determine UAV positions, then a deterministic mobile policy can be obtained, but the discrete value causes imprecise movement control

Engineering Contradiction:

Improvemovement control precisionVSAvoidpolicy evaluation complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent transforms the discrete mobile policy values from Q-learning into continuous values by applying a softmax function. This parameter transformation enables precise movement control while maintaining the benefits of reinforcement learning. The continuous policy values allow for gradient-based optimization and more nuanced control decisions.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent replaces the traditional discrete Q-learning evaluation mechanism with a continuous policy gradient mechanism. By using the policy gradient theorem and continuous value functions, the system substitutes the discrete decision-making process with a continuous optimization approach, achieving higher precision in UAV movement control.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Adaptability or versatility

If UAVs adjust deployment to adapt communication network to user requirements, then communication coverage is improved, but energy consumption increases

Engineering Contradiction:

Improvecommunication coverage adaptabilityVSAvoidUAV energy consumption

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

The patent implements a reinforcement learning framework where UAVs receive feedback in the form of rewards based on their movement decisions and the resulting communication coverage quality. This feedback mechanism enables UAVs to learn optimal movement strategies that balance coverage adaptability with energy consumption, avoiding unnecessary movements while maintaining service quality.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent employs dynamic mobile policy values that are continuously updated based on real-time observation information and reward signals. This dynamic adjustment allows UAVs to adapt their deployment positions flexibly in response to changing user requirements while optimizing energy usage through learned patterns rather than reactive movements.

Inventive Principle:

Principle #15Dynamics

3Ease of operation

If discrete mobile policy is used for UAV control, then implementation is simplified, but movement accuracy deteriorates

Engineering Contradiction:

Improvecontrol implementation simplicityVSAvoidUAV position control accuracy

Core Design Contradiction:

Ease of operationVSMeasurement precision

Solution Approach 1:

The patent changes the parameter space of mobile policy from discrete to continuous by applying the softmax function to Q-learning outputs. This transformation maintains the simplicity of policy implementation while dramatically improving position control accuracy, as continuous values provide finer granularity for movement decisions.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent transitions the policy evaluation from a discrete dimension to a continuous dimension. By mapping discrete Q-values to continuous policy probabilities through softmax, the system adds the dimension of continuous variation, enabling precise control while preserving the computational tractability of the original discrete framework.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Data Source

PatentUS11161609B2Multi-UAV continuous movement control method, apparatus, device, and storage medium for energy efficient communication coverage

Publication Date: 2021.11.02 BEIHANG UNIV

AI summary

Provided is a multi-UAV continuous movement control method for energy efficient communication coverage. The method includes: determining observation information at a current moment, the observation information including one or more of the following: energy consumption information of a UAV, coverage indication information of users covered by a UAV network facilitated by the UAV or coverage fairness information of the UAV network; determining control information corresponding to the observation information using a DDPG model according to the observation information, wherein the DDPG model is generated from a DDPG algorithm with sample information as an input, the sample information indicating a mapping relationship between sampled observation information and sampled control information; and controlling the UAV to move according to the control information. The value of the mobile policy from the DDPG model is continuous, thus mitigating the imprecision in existing movement control due to the discreteness in evaluating the mobile policy.