Distributed Power Allocation in MMWave Cellular Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In mmWave wireless communication networks, centralized control leads to scalability issues due to sub-optimal solutions from non-convex optimization, and the stationarity assumption is violated in multi-agent settings, causing interference and performance hindrances.
Innovation Solution
A multi-agent deep reinforcement learning-enabled distributed scheme is employed, where each base station acts as an actor network to determine transmit power, a critic network to evaluate quality functions based on local and neighboring base stations' observations, and a decentralized training unit to train quality functions over neighboring base stations, using a distributed version of the MADDPG algorithm to address non-stationarity and reduce communication overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If centralized control is used for power allocation, then coordination and interference management are improved, but scalability deteriorates due to non-convex optimization complexity
Solution Approach 1:
The centralized power allocation problem is segmented into multiple distributed agent decisions. Each base station agent independently determines its transmit power based on local observations and learned policies, eliminating the need for centralized optimization while maintaining coordination through the multi-agent reinforcement learning framework.
Solution Approach 2:
The critic network serves as an intermediary that evaluates the joint quality function of multiple agents. It receives observations from multiple base station agents and provides gradient information back to each agent, enabling coordinated decision-making without centralized control.
2Device complexity
If multi-agent reinforcement learning is used for distributed power allocation, then scalability is improved, but training stability deteriorates due to violation of stationarity assumption
Solution Approach 1:
The critic network provides feedback to each agent by evaluating the joint quality function and computing gradients with respect to each agent's actions. This feedback mechanism allows agents to adjust their policies based on the overall system state, stabilizing training despite the non-stationary environment created by other learning agents.
Solution Approach 2:
The quality function of multiple agents is merged into a single joint quality function evaluated by the critic network. This unified evaluation approach allows the system to consider the collective impact of all agents' actions, stabilizing the learning process by providing a consistent objective for all agents to optimize.
3Reliability
If conventional centralized methods like WMMSE and FP are used, then power allocation performance is improved, but communication overhead increases
Solution Approach 1:
Each base station agent independently determines its transmit power using local observations and learned policies, without requiring centralized computation or extensive information exchange. The agents self-organize the power allocation process, dramatically reducing communication overhead while maintaining performance through the distributed multi-agent reinforcement learning framework.
Data Source
AI summary
A base station associated with user devices in a wireless network includes a plurality of base stations. The base station includes a processor and a memory including instructions that, when executed by the processor, cause the base station to function as an actor network configured to determine a current transmit power, a critic network configured to evaluate a quality function of previous transmit powers of the base station based on local observations and previous transmit powers of neighboring base stations, and a decentralized training unit configured to train the quality function over the neighboring base stations. The neighboring base stations are a subset of the plurality of base stations, and the current transmit power is determined based on the previous transmit powers of the base station, direct channel gains between the base station and the user devices, and interference measures from the user devices.