A Spatiotemporal Graph Network Multi-Agent Reinforcement Learning Method for UAV Routing

By employing a multi-agent reinforcement learning method in spatiotemporal graph networks, the problems of routing information lag and inconsistent parameter updates in highly dynamic scenarios of UAV ad hoc networks were solved, achieving stable next-hop selection and parameter updates, and improving the throughput, latency, and energy balance of UAV routing.

CN122138235APending Publication Date: 2026-06-02GUANGDONG UNIV OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
GUANGDONG UNIV OF TECH
Filing Date
2026-03-18
Publication Date
2026-06-02

AI Technical Summary

Technical Problem

Existing UAV ad hoc network routing protocols suffer from problems such as delayed routing information, high control overhead, frequent link breaks, and difficulty in timely congestion avoidance in highly dynamic scenarios. Furthermore, existing deep learning and reinforcement learning methods fail to effectively characterize the evolution of link and queue load over time, leading to unstable decision-making and inconsistent parameter updates.

Method used

A spatiotemporal graph network multi-agent reinforcement learning method is adopted. Local observation data packets are generated through data collection, normalization and encapsulation. A time-varying directed topology is constructed and graph attention space aggregation is performed. Combined with a long short-term memory network and an actor policy network, the next hop selection is generated and distributed forwarding is performed to form a parameter update closed loop.

Benefits of technology

It achieves stable decision-making and consistent parameter updates for UAV routing in highly dynamic scenarios, improving throughput, latency and energy balance, and solving the problems of decision fluctuation and inconsistent parameter updates in existing technologies.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122138235A_ABST
    Figure CN122138235A_ABST
Patent Text Reader

Abstract

This invention relates to the field of routing in wireless communication networks, and more particularly to a multi-agent reinforcement learning method for UAV routing in spatiotemporal graph networks. The method includes: collecting local state and link measurement data of nodes to generate local observation packets; determining neighbors, constructing a time-varying directed local subgraph, and encapsulating node and link features to generate a subgraph feature packet; aggregating node and link features through a graph attention network to generate spatial embeddings; performing temporal fusion of the spatial embedding sequence based on a sliding time window and a long short-term memory network to generate a spatiotemporal state representation packet; selecting the next hop through a policy network inference and performing distributed forwarding to generate a routing log packet; and performing experience replay, multi-index reward calculation, and gradient update at the central node based on the logs to generate and synchronously update network parameters. This invention can adapt to dynamic topologies, capture spatiotemporal dependencies, and achieve continuous optimization of global network performance through a framework of centralized training and distributed execution.
Need to check novelty before this filing date? Find Prior Art