Q-Learning Network Agent for Wireless Cell Assignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing wireless network cell assignment methods are often suboptimal due to reliance on static rules and thresholds, leading to inefficient resource allocation and performance issues as radio and traffic conditions change over time.
Innovation Solution
Implementing a network agent that uses a Q-learning function to estimate the future effects of assigning wireless devices to candidate cells based on current network conditions, allowing for proactive and predictive cell assignments that maximize network performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If static rules and thresholds are used for cell assignment, then the system is simple to implement, but network performance and resource allocation efficiency deteriorate under changing conditions
Solution Approach 1:
The patent applies dynamics by transitioning from static cell assignment rules to a dynamic Q-learning based system. The network agent continuously learns and adapts cell assignment decisions based on changing radio conditions and traffic patterns, making the system responsive to environmental changes while maintaining implementation feasibility through iterative learning.
Solution Approach 2:
The patent implements feedback mechanisms where the network agent receives performance measurements from the wireless network, processes this feedback information, and uses it to update its Q-learning model. This closed-loop feedback enables the system to continuously improve network performance by learning from past assignments and adjusting future decisions accordingly.
2Stability of the object's composition
If reactive cell assignment based on predefined rules is used, then the system is stable and predictable, but adaptability to changing radio and traffic conditions deteriorates
Solution Approach 1:
The patent applies preliminary action by having the network agent proactively learn optimal cell assignment strategies before conditions change. The Q-learning model continuously updates its knowledge base in advance, enabling the system to make informed decisions when conditions change rather than merely reacting to changes after they occur.
Solution Approach 2:
The patent implements self-service through the autonomous Q-learning network agent that automatically adapts to changing conditions without requiring manual reconfiguration. The system serves itself by continuously learning from network measurements and independently optimizing cell assignments, maintaining both stability through learned patterns and adaptability through continuous learning.
3Productivity
If machine learning based cell assignment is implemented, then network performance and resource allocation efficiency improve, but system complexity increases
Solution Approach 1:
The patent introduces an intermediary network agent that acts as a mediator between the complex Q-learning algorithm and the existing wireless network infrastructure. This intermediary layer handles the complexity of machine learning internally while presenting simple, standardized interfaces to the network, thereby improving performance without proportionally increasing overall system complexity.
4Loss of time
If initial cell assignment is made at attachment time, then the process is quick and simple, but optimality of resource allocation deteriorates as conditions change over time
Solution Approach 1:
The patent applies continuity of useful action by implementing continuous cell assignment optimization rather than one-time initial assignment. The Q-learning network agent continuously monitors network conditions and recalculates optimal assignments over time, maintaining assignment optimality without significantly increasing the time loss through efficient incremental learning and decision-making.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
A method and a network agent (200) for providing cell assignment for a wireless device (202) served by a network node (204). An input vector is created for a set of candidate cells based on measurements (2:1A, 2:1B) by the wireless device and/or by the network node related to performance and signals. A future effect of assigning the wireless device to a candidate cell is estimated (2:3) for each candidate cell by applying the created input vector to an effect estimation function which may be a Q-learning function. A cell in the set of candidate cells is then determined and assigned (2:4) for the wireless device, based on the estimated future effects of the candidate cells. The cell that provides the best future effect may be selected for cell assignment.