Model training task scheduling method and device, electronic equipment and storage medium

Abstract

An embodiment of the invention discloses a model training task scheduling method and device, electronic equipment and a storage medium, relates to the technical field of computers, and can effectivelyimprove the model training efficiency. The model training task scheduling method comprises the steps of: acquiring data set attribute information of a target training data set corresponding to a to-be-executed target model training task, wherein the data set attribute information comprises an identifier of the target training data set; according to the data set attribute information, determiningtask statistics information of other model training tasks using the target training data set to perform model training, wherein the task statistics information comprises distribution information and / or operation state information of the other model training tasks in each server in a cluster; and selecting a target server from the servers according to the task statistical information, and scheduling the target model training task to the target server for execution. The model training task scheduling method is suitable for model training of machine learning.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a scheduling method, device, electronic equipment and storage medium for model training tasks. Background technique [0002] In recent years, artificial intelligence technology has been more and more widely used in industry and life. As an important branch of artificial intelligence, machine learning can obtain ideal mathematical models through a large amount of training data. [0003] However, due to the huge amount of data required for model training, often tens of millions of files, it often takes a lot of time to read the training data during model training, resulting in low model training efficiency. Contents of the invention [0004] In view of this, embodiments of the present invention provide a method, device, electronic device, and storage medium for scheduling a model training task, which can effectively improve model training efficiency. [0005] In the f...

AI Technical Summary

Problems solved by technology

[0003] However, due to the huge amount of data required for model training, often tens of millions of files, it often takes a lot of time to read the training data during model training, resulting in low model training efficiency

Images