SD-WAN Client Device Routing via Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In software-defined wide area networks (SD-WANs), determining the most efficient network path for client devices to access network services is challenging due to dynamic changes and the unpredictability of network paths, especially when parts of the path traverse the Internet, making it difficult to select the best entry point for optimal performance.
Innovation Solution
A client device uses active and passive network probing combined with reinforcement learning algorithms, specifically multi-armed bandit algorithms, to determine the best entry point by calculating expected rewards based on network performance data, reducing the number of path switches and improving network awareness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple geographically dispersed entry points are used to improve service performance for globally distributed client devices, then service accessibility and performance are improved, but network path unpredictability and selection complexity increase
Solution Approach 1:
The patent implements dynamic network path selection by continuously monitoring network conditions (latency, packet loss, jitter) and adapting entry point selection in real-time. The system transitions from static geographic-based selection to dynamic performance-based selection, allowing the network to adapt to changing conditions and maintain optimal performance despite path variability.
Solution Approach 2:
The system employs feedback mechanisms by measuring actual network performance metrics for each available entry point and using this information to inform subsequent routing decisions. The client device receives feedback about network conditions and adjusts its entry point selection accordingly, creating a closed-loop control system that optimizes performance.
2Ease of operation
If traditional manual configuration or simple threshold-based path selection is used, then configuration simplicity is maintained, but network performance optimization is insufficient due to inability to definitively measure and determine best network path
Solution Approach 1:
The system implements self-service by enabling client devices to autonomously measure and evaluate network paths without requiring manual configuration or centralized control. Each client device independently probes available entry points, measures performance metrics, and selects the optimal path based on its own observations, eliminating the need for complex manual setup while achieving precise performance measurement.
Solution Approach 2:
The patent applies preliminary action by having client devices perform proactive network probing and performance measurement before actual data transmission begins. The system pre-evaluates available entry points by sending test packets and measuring response characteristics, allowing informed routing decisions to be made before production traffic flows, thus optimizing performance from the outset.
3Measurement precision
If active network probing is performed to measure and determine the best network path, then network path selection accuracy is improved, but data traffic used for probing and time for measurements increase
Solution Approach 1:
The system applies partial action by performing probing on a selective basis rather than continuously monitoring all possible entry points. Client devices probe a subset of available entry points or use simplified measurement techniques when conditions permit, reducing the overhead of continuous comprehensive probing while still maintaining adequate path selection accuracy. The system balances measurement thoroughness with resource conservation.
Data Source
AI summary
An example client device includes processing circuitry and a memory including instructions that, when executed by the processing circuitry, cause the client device to undertake certain actions. Certain instructions cause the device to periodically measure active network performance data for a network, calculate expected rewards for the plurality of entry points, select an expected best entry point based on the expected rewards, route data to the selected entry point, measure passive network performance data for the selected entry point, and update a reinforcement learning algorithm, based in part on the measured passive network performance data.