Deep Reinforcement Learning for Wireless Network Cell Parameter Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional wireless network optimization methods face challenges in achieving lower optimization costs, shorter optimization times, and higher accuracy as networks become more complex, requiring significant resources and time, and are not adaptable for one-shot optimization across different networks.
Innovation Solution
The implementation of deep reinforcement learning (DRL) techniques to train neural networks for selecting actions to adjust cell parameters in wireless networks, using experience tuples and expert experience to improve training efficiency and accuracy, allowing for one-shot optimization without the need for extensive site visits or accurate engineer parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional wireless network optimization methods are used, then optimization can be performed with existing techniques, but optimization time and cost increase significantly as networks become more complex
Solution Approach 1:
The patent replaces conventional mechanical optimization methods with deep reinforcement learning (DRL) techniques. The DRL agent learns optimal network configurations through trial and error in a simulated environment, substituting traditional manual or algorithmic optimization approaches with intelligent agents that can adapt to network complexity without proportionally increasing optimization time
Solution Approach 2:
The patent performs preliminary training of DRL agents in simulated environments before deploying them to real networks. Experience tuples are generated and stored in advance, allowing the system to have pre-learning optimization knowledge ready before actual network optimization is needed, thus reducing real-world optimization time
2Reliability
If conventional optimization methods are used, then existing tools can be utilized, but optimization cost increases with network complexity
Solution Approach 1:
The patent substitutes expensive conventional optimization tools and processes with DRL-based optimization. The DRL agent requires minimal human intervention and can operate autonomously, reducing labor costs and tool licensing fees associated with traditional optimization methods while maintaining or improving accuracy
Solution Approach 2:
The patent creates virtual copies of network environments for training DRL agents. By using simulated network copies instead of real networks during the training phase, the system avoids the high costs of trial-and-error optimization in production environments, while the learned policies can be applied to real networks
3Adaptability or versatility
If conventional optimization methods are used, then standard procedures can be followed, but adaptability to different networks is limited
Solution Approach 1:
The patent implements dynamic adaptation through DRL agents that continuously learn from experience tuples generated during training and operation. The agents can adapt to different network configurations, topologies, and conditions by learning new policies, making the optimization method highly versatile across various network types without requiring manual reconfiguration
Solution Approach 2:
The patent develops universal DRL agents that can optimize multiple types of wireless networks (LTE, 5G, etc.) and handle various optimization scenarios (coverage, capacity, energy efficiency) using the same underlying framework. The experience tuple structure and training methodology are generalizable across different network contexts
4Loss of time
If one-shot optimization is implemented using DRL, then optimization time is reduced, but training data requirements increase
Solution Approach 1:
The patent performs preliminary generation of experience tuples in simulated environments before actual optimization is needed. By pre-training DRL agents with synthetic training data that mimics real network conditions, the system reduces the need for extensive real-world data collection while enabling rapid one-shot optimization deployment
Solution Approach 2:
The patent uses virtual copies of network environments to generate training data. Instead of collecting massive amounts of real network data, the system creates synthetic experience tuples from simulated networks, reducing data acquisition costs and enabling efficient training without proportionally increasing data volume requirements
Data Source
Figure 1~2
Figure 3~5
Figure 6
AI summary
A neural network is trained using deep reinforcement learning (DRL) techniques for adjusting cell parameters of a wireless network by generating a plurality of experience tuples, and updating the neural network based on the generated experience tuples. The trained neural network may be used to select actions to adjust the cell parameters. Each experience tuple includes a cell identifier, a first state, a second state, an action applied to the cell that moves the cell from the first state to the second state, a local reward, and a global reward. The neural network is updated based on whether or not each action is acceptable, which is determined based on the global reward and the local reward associated with each action.