DQN Flow Scheduling for Distributed Computing Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current distributed computing network coordinate flow scheduling methods fail to optimize completion time due to neglect of dependency relationships between flows and reliance on heuristic algorithms that provide only approximate solutions, leading to suboptimal performance.
Innovation Solution
A DQN-based distributed computing network coordinate flow scheduling method that uses a deep reinforcement learning intelligent agent to establish environmental features, including congestion situations and dependency relationships, to generate executable scheduling policies that minimize overall elapsed time, with a policy mapper and gainer to iteratively optimize the agent's actions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If heuristic algorithms are used to simplify the scheduling problem, then the computational complexity is reduced, but only approximate optimal solutions can be obtained and algorithm performance is limited
Solution Approach 1:
The patent transforms the scheduling problem from a deterministic optimization problem into a reinforcement learning problem by changing the approach parameters: using state-space representation, action-space definition, and reward functions to guide the DQN agent toward optimal scheduling decisions without requiring exhaustive computation of all possible schedules
Solution Approach 2:
The patent replaces traditional heuristic algorithms with a deep reinforcement learning system (DQN). The mechanical system of rule-based heuristic decision-making is substituted with a neural network-based intelligent agent that learns optimal scheduling policies through interaction with the environment, achieving better performance without proportional increases in computational complexity
2Ease of operation
If coordinate flow scheduling is performed only at the data stream level, then the scheduling process is simplified, but the completion time at the entire computing task level cannot be optimized
Solution Approach 1:
The patent extends the scheduling scope from a single dimension (data stream level) to multiple dimensions by simultaneously considering coordinate flows across the entire computing task. The state-space representation captures both individual coordinate flow states and their interdependencies within the broader task context, enabling optimization at multiple hierarchical levels
Solution Approach 2:
The patent segments the scheduling problem into manageable components: state-space representation, action-space definition, and reward function design. This segmentation allows the complex multi-level scheduling problem to be solved through systematic reinforcement learning while maintaining tractability
Data Source
AI summary
The present application discloses a DQN-based distributed computing network coordinate flow scheduling system and method. The method includes: establishing environmental feature data based on distributed computing task information and a congestion situation of a port queue in a programmable forwarding platform on a data plane, establishing and training a deep reinforcement learning intelligent agent based on an action value network and a target network in DQN, and the deep reinforcement learning intelligent agent outputting abstract actions; receiving, by a policy mapper, the abstract actions and mapping them into an executable coordinate flow scheduling policy; executing, by the programmable forwarding platform, the executable coordinate flow scheduling policy and updating the congestion situation of the port queue; and recording, a policy gainer, a completion time of a distributed computing task as a real-time reward of the deep reinforcement learning intelligent agent and iteratively optimizing the deep reinforcement learning intelligent agent.