Multi-Agent Deep Reinforcement Learning for Sub-THz UDN Hybrid Beamforming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for interference control and hybrid beamforming in mobile communication networks face challenges with high complexity, limited data rate optimization, and inefficiencies due to the use of single-agent deep reinforcement learning and single-antenna systems, especially in high-density networks with multiple base stations and users.
Innovation Solution
A multi-agent deep reinforcement learning approach is introduced, utilizing channel state information to perform interference and antenna gain-based reinforcement learning, searching for optimal analog beamforming matrix pairs, and applying signal-to-leakage plus noise ratio (SLNR) maximization techniques to optimize transmission power and minimize interference between user equipment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single-agent deep reinforcement learning technique is applied for interference control, then the system complexity is reduced, but the system cannot scale effectively when the number of base stations and users increases
Solution Approach 1:
The patent divides the single-agent reinforcement learning system into multiple independent agents, where each base station operates as an independent agent making local decisions. This segmentation allows the system to scale with the number of base stations and users while maintaining manageable complexity through distributed control.
2Device complexity
If a single antenna system is applied to a user, then the hardware complexity is reduced, but spatial multiplexing gain cannot be obtained
Solution Approach 1:
The patent transitions from single-antenna to multi-antenna systems, adding the spatial dimension to enable spatial multiplexing. By employing multiple antennas at both base stations and users, the system achieves higher data rates through spatial diversity and multiplexing gains while maintaining practical hardware configurations.
3Device complexity
If a hybrid beamforming system is not applied, then the circuit complexity is reduced, but high data rate and maximum transmission speed per unit area cannot be obtained
Solution Approach 1:
The patent combines analog beamforming and digital beamforming into a hybrid beamforming system. This merging allows the system to achieve the spatial multiplexing gain of digital beamforming while maintaining the circuit simplicity of analog beamforming, thereby obtaining high data rates without excessive circuit complexity.
4Loss of time
If beam pairs are greedily selected without artificial intelligence technology, then the computation time is reduced, but residual interference between users cannot be optimally controlled
Solution Approach 1:
The patent implements reinforcement learning with feedback mechanisms where agents learn from the consequences of their actions. The multi-agent reinforcement learning algorithm iteratively optimizes beam pair selections by considering interference effects, enabling optimal control of residual interference between users while managing computation time through efficient learning processes.
Data Source
AI summary
Disclosed is a method and system for interference control and hybrid beamforming using multi-agent deep reinforcement learning for multiple users. The disclosed method includes performing multi-agent reinforcement learning using interference and antenna gain information that is expected based on a gain table designed using channel state information (CSI) of all user equipments; searching for an analog beamforming matrix pair corresponding to a link that maximizes antenna gain and minimizes interference between user equipments through multi-agent reinforcement learning; applying a signal-to-leakage plus noise ratio (SLNR) maximization technique that minimizes the interference between the user equipments based on the link; and optimizing transmission (Tx) power for each link based on iterative water-filling.