Job scheduling system of distributed training cluster

The distributed training cluster job scheduling system solves the problem of insufficient resource utilization in large-scale distributed artificial intelligence training scenarios, realizes efficient scheduling and stable operation of training jobs, and improves resource utilization and training efficiency.

CN120803653APending Publication Date: 2025-10-17GUANGZHOU XIAOPENG CONNECTIVITY TECH CO LTD
0 Cites 0 Cited by

Patent Information

Application Number
CN202510926563.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-07-04
Publication Date
2025-10-17

AI Technical Summary

Technical Problem

Existing scheduling systems struggle to efficiently schedule terabyte-scale training data in large-scale distributed AI training scenarios, leading to underutilization or overuse of resources and making it difficult to guarantee the stable operation and efficient execution of training jobs.

Method used

A job scheduling system for a distributed training cluster is provided, including queue management service, job deployment management service, job analysis service, job termination management service and fault tolerance service. Through these services, training jobs are scheduled in a reasonable manner to ensure efficient use of resources and stable operation of jobs.

Benefits of technology

It enables efficient scheduling of TB-level training data, ensuring the reliability and availability of training jobs, improving training efficiency and effectiveness, and reducing resource waste and energy consumption.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a distributed training cluster job scheduling system which comprises a queue management service, a job queue, a job deployment management service, a job analysis service, a job termination management service and a fault-tolerant service. The number of the job queues is multiple. According to the method, a plurality of queues are managed for a large-scale distributed AI training scene, the training jobs are analyzed, corresponding processing is adaptively executed, the training jobs are reasonably scheduled, it is ensured that the training jobs have high availability and reliability, and the training efficiency and the training effect can also be ensured when a large number of training jobs exist.
Need to check novelty before this filing date? Find Prior Art