Load Balancing via Transfer Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing load balancing techniques in cellular communication systems face inefficiencies due to uneven distribution of devices across base stations, leading to sub-optimal interference levels and throughput, particularly when some base stations are heavily loaded while others are lightly loaded, and require extensive training data for AI algorithms to optimize device redistribution.
Innovation Solution
The Transfer Reinforcement Learning Load Balancing (TRLLB) algorithm combines deep learning, reinforcement learning, and transfer learning to efficiently train AI algorithms by reusing policies learned from source tasks and updating them with recent experiences, formulating load balancing for idle mode users as a Markov decision process to improve data efficiency and system throughput.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If reinforcement learning algorithms are used to learn control policy for load balancing, then device distribution optimization is achieved, but large amount of training data is required
Solution Approach 1:
The patent applies preliminary action by pre-training reinforcement learning agents in source environments (simulated cellular systems) before deploying them to target environments. The source policies are learned in advance through simulation, allowing the agent to acquire useful knowledge without requiring extensive training data in the actual target system. This pre-learning phase prepares the policy beforehand, reducing the data burden during real-world deployment.
Solution Approach 2:
The patent uses copying by creating simulated source environments that replicate the characteristics of target cellular systems. Multiple source policies are trained in these copied virtual environments, and the best performing policy is selected and transferred to the target system. This copying approach allows extensive training to occur in simulated copies rather than requiring equivalent training data in the actual target system.
2Reliability
If multiple source policies are evaluated and selected based on reward comparison, then optimal policy transfer is achieved, but additional evaluation computations are required
Solution Approach 1:
The patent applies partial action by evaluating only a limited number of pre-trained source policies (e.g., top-performing ones) rather than exhaustively testing all possible policies. The method selectively applies and compares a subset of source policies against the target policy, achieving sufficient reliability without requiring complete evaluation of every possible policy variant. This selective evaluation reduces computational complexity while maintaining effective policy transfer.
Data Source
AI summary
Rapid and data-efficient training of an artificial intelligence (AI) algorithm are disclosed. Ground truth data are not available and a policy must be learned based on limited interactions with a system. A policy bank is used to explore different policies on a target system with shallow probing. A target policy is chosen by comparing a good policy from the shallow probing with a base target policy which has evolved over other learning experiences. The target policy then interacts with the target system and a replay buffer is built up. The base target policy is then updated using gradients found with respect to the transition experience stored in the replay buffer. The base target policy is quickly learned and is robust for application to new, unseen, systems.