Distributed Network Scheduling for Faster Multi-Cell Policy Training
Solution Overview
Problem
Conventional centralized scheduling solutions for wireless networks face challenges in multi-cell and multi-level heterogeneous networks due to large action spaces and slow neural network training convergence, making them infeasible for practical deployment.
Innovation Solution
Implementing a distributed scheduling method using multiple agents, where each network device obtains local state information, generates scheduling information based on a corresponding sub-policy network, and iteratively trains to optimize scheduling policies, enhancing system performance through distributed policy networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a centralized scheduling solution is used with a unique agent responsible for all cells/networks, then scheduling decisions can be made globally optimally, but the action space becomes excessively large and training convergence becomes extremely slow
Solution Approach 1:
The patent divides the centralized scheduling problem into multiple distributed sub-problems by creating separate agents for different cells or network levels. Each agent manages scheduling decisions for its specific domain, reducing the action space from global to local level. This segmentation enables parallel training of multiple smaller agents instead of one large agent, significantly improving training convergence speed while maintaining effective scheduling capabilities.
2Reliability
If a centralized scheduling solution is used with a unique agent responsible for all cells/networks, then scheduling decisions can be made globally optimally, but the complexity of the system increases making deployment infeasible
Solution Approach 1:
The system is segmented into multiple independent agents, each responsible for a specific cell or network level. This distribution of functionality reduces the complexity burden on any single component, making the overall system more manageable and feasible for deployment in multi-cell and multi-level heterogeneous networks.
Solution Approach 2:
Each distributed agent independently makes scheduling decisions for its own domain without requiring centralized coordination. This self-service capability reduces the complexity of inter-agent communication and coordination, simplifying the overall system architecture while maintaining effective local optimization.
3Productivity
If the action space is reduced by using distributed agents, then training convergence speed improves, but the ability to make globally optimal decisions may be compromised
Solution Approach 1:
While agents are distributed for training efficiency, the patent employs mechanisms to merge and coordinate their decisions. Through shared experience buffers, centralized value networks, or inter-agent communication protocols, the system combines local decisions into globally coherent scheduling policies, ensuring that distributed training does not sacrifice overall optimization capability.
Solution Approach 2:
The system implements feedback mechanisms where agents receive performance metrics and adjust their policies accordingly. This feedback loop ensures that even with reduced action spaces, agents learn to make decisions that contribute to global optimization goals, maintaining scheduling decision quality while benefiting from faster convergence.
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
Embodiments of this application provide a scheduling method, a scheduling algorithm training method, a related system, and a storage medium. The scheduling method is applied to a scheduling control system, the scheduling control system includes K network devices, K is an integer greater than 1, and the method includes: A first network device obtains target state information of a first area, where the target state information includes network state information and user data packet cache information; the first network device generates first scheduling information based on the target state information of the first area and a first target sub-policy network, where the first target sub-policy network is a target sub-policy network that is in K target sub-policy networks and that corresponds to the first network device; and the first network device delivers the first scheduling information to a terminal in the first area. In this solution, performance of the scheduling control system is improved. In addition, feasibility of a scheduling control solution is improved by deploying policy networks in a distributed manner.