Deep Reinforcement Learning for Multi-Access Traffic Management
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current multi-access traffic management technologies lack efficient strategies to meet diverse quality-of-service (QoS) requirements in multi-access communications networks, relying on pre-defined policies and mathematical models that are not scalable or adaptable to dynamic network conditions.
Innovation Solution
The implementation of a deep reinforcement learning (DRL) architecture at edge compute nodes for multi-access traffic management, utilizing model-free approaches that learn optimal traffic strategies through interactions with the environment, incorporating observations from user equipment and radio access networks to dynamically adjust traffic steering and splitting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If pre-defined policies and mathematical models are used for traffic management, then implementation is straightforward, but the system lacks adaptability to dynamic network conditions
Solution Approach 1:
The patent implements dynamic traffic management by transitioning from static pre-defined policies to reinforcement learning agents that continuously adapt their decision-making based on real-time network conditions. The agents learn optimal traffic distribution strategies dynamically by interacting with the multi-access network environment and receiving feedback on QoS outcomes.
Solution Approach 2:
The reinforcement learning agents autonomously manage traffic distribution without requiring manual configuration or external control. The system self-learns optimal strategies through trial and error interactions with the network environment, automatically adjusting traffic steering decisions to meet diverse QoS requirements.
2Productivity
If deep reinforcement learning architecture is implemented, then adaptability and QoS optimization improve, but computational complexity and resource requirements increase
Solution Approach 1:
The patent segments the traffic management function into multiple independent reinforcement learning agents, each responsible for specific traffic flows or access technologies. This modular approach distributes computational complexity across multiple agents rather than requiring a single complex centralized system, making the overall solution more manageable and scalable.
Solution Approach 2:
The reinforcement learning agents act as intermediary components between the network infrastructure and traffic flows. They translate complex network conditions into simplified state representations and convert traffic management decisions into actionable steering commands, bridging the gap between raw network data and control actions.
3Adaptability or versatility
If model-free reinforcement learning is used, then the system can handle diverse QoS requirements, but training time and initial setup resources increase
Solution Approach 1:
The patent performs preliminary training of reinforcement learning agents using simulated network environments before deploying them to production systems. This offline training phase allows agents to learn optimal strategies without impacting real network performance, reducing the time penalty associated with model-free learning when the system is actually operational.
Solution Approach 2:
The patent creates virtual copies of the network environment through simulation platforms that replicate real multi-access network conditions. These simulated environments serve as training grounds for reinforcement learning agents, allowing extensive experimentation and learning without consuming real network resources or affecting actual service quality.
Data Source
AI summary
The present disclosure is related to multi-access traffic management in edge computing environments, and in particular, artificial intelligence (AI) and/or machine learning (ML) techniques for multi-access traffic management. A scalable AI/ML architecture for multi-access traffic management is provided. Reinforcement learning (RL) and/or Deep RL (DRL) approaches that learn policies and/or parameters for traffic management and/or for distributing multi-access traffic through interacting with the environment are also provided. Deep contextual bandit RL techniques for intelligent traffic management for edge networks are also provided. Other embodiments may be described and/or claimed.