Self-Learning Network Control Policy Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for generating network control policies are limited by their scalability and complexity, requiring high computational costs and manual configuration, making them inefficient for dynamic network adjustments and unable to adapt to changing network conditions.
Innovation Solution
A self-learning method and apparatus for generating network control policies that utilize a central control server, prediction server, and training server to predict network status and adjust policies in real-time, optimizing data flow routing and improving network control efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional optimization methods (linear planning or integer planning) are used to generate network control policy, then a network control policy can be obtained, but the calculation complexity increases and scalability becomes weak as network elements, service types, and traffic increase
Solution Approach 1:
The patent replaces conventional mechanical optimization methods (linear planning, integer planning) with a deep reinforcement learning model. The neural network learns optimal routing policies through self-training on historical data, substituting complex mathematical optimization with a trained intelligent agent that can generalize to new scenarios without re-optimization.
Solution Approach 2:
The system performs preliminary action by pre-training the reinforcement learning model on historical network data offline. This allows the model to learn optimal policies in advance, so when deployed online, it can make rapid decisions without performing complex real-time optimization calculations.
2Reliability
If offline optimization is used to generate network control policy, then a control policy can be obtained, but dynamic instant policy adjustment cannot be implemented
Solution Approach 1:
The patent transforms the static offline optimization approach into a dynamic system using reinforcement learning. The trained model can adapt to changing network conditions in real-time by processing current state inputs and generating updated routing policies, enabling dynamic instant adjustment while maintaining policy quality.
Solution Approach 2:
The system implements feedback by continuously monitoring network state and using the reinforcement learning model to adjust policies based on current conditions. The model learns from historical feedback (routing outcomes) and adapts its policy generation based on real-time network state observations.
3Reliability
If optimization problem is solved again when network status changes, then updated control policy can be obtained, but the process becomes hysteretic and requires large amount of manual configuration
Solution Approach 1:
The patent replaces the time-consuming manual re-optimization process with a pre-trained reinforcement learning model. When network status changes, the model instantly processes the new state through neural network inference (milliseconds) rather than performing full optimization calculations, eliminating both time loss and manual configuration requirements.
Solution Approach 2:
The system achieves self-service by automatically adapting to network changes without human intervention. The reinforcement learning model autonomously processes new network states and generates updated policies, eliminating the need for manual model reconfiguration that would otherwise be required when network topology or traffic patterns change.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention provides a method for generating a network control policy, including: obtaining network status data; estimating network status information according to the network status data, and predicting next-moment status information of the network according to the network status information; receiving a data flow currently entering the network, and preprocessing the data flow, to obtain a preprocessing result of the data flow; generating a control action by using a current network control policy and according to the network status information, the next-moment status information of the network, and the preprocessing result of the data flow; controlling the network according to the control action, and obtaining feedback information of the network controlled according to the control action; and adjusting the current network control policy in real time according to the feedback information, to obtain a next-moment network control policy. The network control policy is adaptively adjusted in real time, to improve network control efficiency.