Cross-Cluster Task Scheduling via Pluggable Algorithm Services
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cross-cluster computing environments face challenges in efficiently allocating computing tasks to maximize resource utilization and energy efficiency while meeting user needs, particularly in high-concurrency and big data scenarios, without disrupting existing services.
Innovation Solution
A task scheduling system and method that integrates multiple scheduling algorithms as independent services, using standardized input and output formats to decouple job request collection, scheduling, and execution modules, allowing seamless integration without modifying existing systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple scheduling algorithms are integrated into a unified system, then task allocation efficiency and resource utilization improve, but system complexity and integration difficulty increase
Solution Approach 1:
The system segments scheduling algorithms into independent, pluggable modules that can be independently developed, tested, and deployed. Each scheduling algorithm operates as a separate service component that communicates through standardized interfaces, eliminating the need for complex monolithic integration while maintaining the benefits of multiple algorithms working together.
Solution Approach 2:
The patent implements a universal scheduling framework that can accommodate multiple different scheduling algorithms through a common interface. This multi-functional architecture allows the same system structure to support various scheduling strategies (e.g., FIFO, round-robin, priority-based) without requiring separate integration processes for each algorithm.
2Productivity
If scheduling algorithms are tightly coupled with the computing environment, then scheduling performance can be optimized, but adaptability to different environments and algorithms decreases
Solution Approach 1:
The patent introduces an intermediary layer (scheduling service module) that sits between the computing environment and the scheduling algorithms. This mediator handles environment-specific details and provides a standardized interface to algorithms, allowing performance optimization for the environment while maintaining algorithm independence and flexibility.
Solution Approach 2:
The system allows dynamic configuration of scheduling parameters and algorithm selection without requiring code changes or tight coupling. By separating algorithm logic from environment-specific parameters, the system can adapt to different computing environments and switch between algorithms by simply changing configuration parameters rather than restructuring the system.
3Adaptability or versatility
If existing services are modified to integrate new scheduling algorithms, then system functionality improves, but service disruption and reliability risks increase
Solution Approach 1:
The patent implements preliminary action by establishing standardized interfaces and protocols before integrating new scheduling algorithms. The framework is prepared in advance with defined communication contracts, allowing new algorithms to be added without modifying existing services. This preliminary structuring ensures that integrations can occur without service disruption.
Solution Approach 2:
The system uses copying by creating independent copies of scheduling algorithm services that can be tested and validated before deployment. New algorithms are instantiated as separate service copies that run alongside existing ones, allowing verification of functionality without affecting the reliability of production services. Failed integrations can be discarded without impacting existing operations.
Data Source
AI summary
The system includes a job request collection and distribution module, a scheduling service module and a job execution service module. The job request collection and distribution module receives first description information of a job to be executed from a user terminal. A current scheduling service module matched with a job scheduling algorithm name in at least one scheduling service module determines a computing resource required by the job to be executed according to the first description information, and then determines a job scheduling result according to the required computing resource and currently available cluster computing resources. The job is submitted to a high-performance computer through a current job execution service module matched with a job execution service name in at least one job execution service module according to a device identifier and a global identifier of the job to be executed contained in the scheduling result.