Network Controller Path Assignment via Deep Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In deterministic networks, path computation faces challenges with flow fragmentation due to the NP-complete nature of assigning new flows, which can lead to system impairments and violate guaranteed delivery times when existing flows are fragmented across multiple paths.
Innovation Solution
A deep reinforcement learning-based approach is implemented to predict traffic flows and assign paths based on resource requirements, using a generative adversarial network to generate synthetic training data and avoid flow fragmentation by optimizing path assignments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If deterministic path computation is used to guarantee packet delivery within bounded time, then delivery time guarantee is improved, but flow fragmentation occurs due to NP-complete nature of flow assignment
Solution Approach 1:
The patent changes the approach from exact deterministic computation to probabilistic machine learning prediction. The deep reinforcement learning agent learns optimal path assignments by training on historical flow data, transforming the NP-complete deterministic optimization problem into a probabilistic prediction problem that avoids fragmentation while maintaining delivery guarantees.
Solution Approach 2:
The patent uses generative adversarial networks to create synthetic training data that copies realistic flow patterns. This synthetic data allows the reinforcement learning agent to be trained on diverse scenarios without requiring actual network traffic, enabling the system to learn fragmentation avoidance strategies in a controlled environment before deployment.
2Productivity
If flows are assigned to multiple paths to utilize network resources, then resource utilization is improved, but flow fragmentation increases causing system impairments
Solution Approach 1:
The patent applies preliminary action by training the deep reinforcement learning agent in advance on synthetic flow data generated by GANs. The agent learns optimal path assignment strategies before actual network operation, enabling it to make fragmentation-free decisions in real-time without compromising either resource utilization or flow integrity during live traffic.
Solution Approach 2:
The reinforcement learning agent uses feedback from its training experiences on synthetic data to continuously improve its path assignment decisions. The agent receives rewards or penalties based on whether its assignments cause fragmentation, allowing it to learn optimal strategies that balance resource utilization with flow integrity through iterative feedback loops.
3Reliability
If deep reinforcement learning is used to predict flows and assign paths, then flow fragmentation is reduced, but computational complexity of path computation increases
Solution Approach 1:
The patent performs the computationally intensive reinforcement learning training in advance using synthetic data, so that during actual network operation, the trained model makes rapid predictions without requiring complex real-time computation. This shifts the computational burden from online path computation to offline training, resolving the contradiction between accuracy and real-time complexity.
Solution Approach 2:
The use of synthetic training data copied from real flow patterns allows the system to train on diverse scenarios without requiring actual network traffic during training. This copying approach enables comprehensive training that improves assignment accuracy while keeping the actual deployment computationally efficient, as the heavy lifting is done on synthetic data.
Data Source
AI summary
In one embodiment, a controller in a network trains a deep reinforcement learning-based agent to predict traffic flows in the network. The controller determines one or more resource requirements for the predicted traffic flows. The controller assigns, using the deep reinforcement learning-based agent, paths in the network to the flows based on the determined one or more resource requirements, to avoid fragmentation of a flow during transmission of the flow through the network. The controller sends, to nodes in the network, assignment instructions that cause the flows to traverse the network via their assigned paths.