Distributed Learning for Radio Resource Allocation With Global Rewards
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing radio resource allocation methods in wireless networks face challenges such as poor performance due to reliance on local information, high communication overhead in centralized approaches, and instability in distributed training, leading to non-convergence and unfair solutions.
Innovation Solution
A decentralized radio resource allocation system using AI, where access points exchange simple messages for policy updates, allowing each access point to learn and adapt policies based on local observations and global rewards, reducing communication overhead and ensuring convergence.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If centralized resource allocation is used, then performance is improved by considering global network state, but communication overhead and latency increase significantly
Solution Approach 1:
The patent segments the centralized resource allocation problem into distributed sub-problems at each access point. Each access point independently learns and executes resource allocation decisions based on local observations, eliminating the need for bulky information exchange while maintaining adaptive performance through localized deep reinforcement learning agents
Solution Approach 2:
The patent introduces a lightweight message-passing mechanism as an intermediary between access points. Each access point exchanges only essential aggregated information (such as total queue length or interference level) with neighbors, enabling coordinated resource allocation without requiring full state information sharing, thus reducing communication overhead while improving global performance
2Reliability
If distributed resource allocation with full information exchange is used, then performance is improved by considering neighboring access points, but communication overhead becomes excessive
Solution Approach 1:
The patent extracts only the most critical aggregated metrics from each access point's local state (such as total queue length, average channel quality, or interference power) and exchanges these extracted features with neighboring access points. This selective extraction enables performance improvement through neighbor awareness while keeping communication complexity low by transmitting only essential summary statistics rather than full state information
3Productivity
If independent distributed training is used at each access point, then scalability is improved, but stability and convergence issues arise leading to unfair solutions
Solution Approach 1:
The patent merges the training processes of distributed access points through a centralized training server that collects experience data from all agents, performs centralized gradient updates, and distributes updated policy parameters back to access points. This centralized training with decentralized execution approach maintains scalability during execution while ensuring training stability and convergence through coordinated updates, preventing unfair solutions by considering global state during the training phase
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A node (201) for a communications network (200) configured to communicate with one or more other nodes (202, 203) in the network and to implement one or more policies for multiple devices (204, 205, 206) in the network which are associated with the node and being configured to: implement (401) a first policy for the multiple devices; determine (402) a local reward or cost for the multiple devices; receive (403) a respective message from each of the one or more other nodes (202, 203) in the network (200) indicating corresponding local rewards or costs of devices (207, 208, 209, 210) associated with each of the other nodes (202, 203); in dependence on the message(s), determine (404) a global reward or cost for the network (200); and update (405) the first policy in dependence on the global reward or cost. This may enable decentralized resource allocation or scheduling in networks.