Task scheduling method and device for training cluster, electronic device, computer readable storage medium and computer program product

By constructing a communication quality matrix and selecting target worker nodes, the problems of low scheduling success rate and uneven resource utilization in the training cluster were solved, achieving efficient training task execution and resource matching, and improving overall training performance.

CN120872552BActive Publication Date: 2025-12-09SHANGHAI ORIENTAL COMPUTER TECHNOLOGY CO LTD
2 Cites 0 Cited by

Patent Information

Application Number
CN202511368911.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-09-23
Publication Date
2025-12-09
Estimated Expiration
2045-09-23

AI Technical Summary

Technical Problem

Existing training cluster scheduling mechanisms suffer from low scheduling success rates and uneven resource utilization in multi-node parallel training scenarios, especially when inter-node communication is inefficient, leading to low execution efficiency of training tasks.

Method used

By analyzing the computational unit requirements and communication quality parameters of the training task, a communication quality matrix is ​​constructed, and target working nodes are selected for task allocation to ensure matching of computational resources and efficient communication, thereby reducing data transmission latency between nodes.

Benefits of technology

It improved the efficiency and success rate of training tasks, reduced data transmission latency between nodes, and enhanced resource utilization.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The application provides a task scheduling method and device of a training cluster, an electronic device, a computer readable storage medium and a computer program product, which are applied to a scheduling node in the training cluster. The training cluster further includes a plurality of working nodes, and each working node includes at least one computing unit. The method includes: analyzing a training task of a machine learning model to obtain a plurality of task instances of the training task and a first number of computing units required by each task instance; receiving a communication quality parameter sent by the working node, and determining a communication quality matrix based on the communication quality parameter; selecting a target working node from the working nodes based on the communication quality matrix and the first number; and scheduling the task instances to the target working node, so that the plurality of target working nodes execute the plurality of task instances, wherein the plurality of target working nodes exchange execution results of the task instances through a communication connection therebetween. Through the application, the execution efficiency and success rate of the training task can be improved.
Need to check novelty before this filing date? Find Prior Art