Distributed training method and device of model, equipment and storage medium

Abstract

The invention provides a distributed training method and device of a model, equipment and a storage medium, and relates to the technical field of artificial intelligence, in particular to the fields of deep learning, cloud computing and the like. According to the specific implementation scheme, an initial segmentation strategy is generated based on a to-be-trained model; determining first attribute description information of the to-be-trained model under the initial segmentation strategy; the first attribute description information is used for representing at least one of the storage space occupation condition and the operation duration of the to-be-trained model under the initial segmentation strategy; based on the first attribute description information, optimizing the initial segmentation strategy to obtain a target segmentation strategy meeting a preset condition; and segmenting the to-be-trained model based on the target segmentation strategy to obtain a segmentation result, the segmentation result being used for performing distributed training on the to-be-trained model. According to the technology disclosed by the invention, for a distributed training scene of the model, the training time is shortened, the training efficiency is improved, and the training cost is reduced.

Description

technical field [0001] The present disclosure relates to the field of artificial intelligence technology, in particular to the fields of deep learning and cloud computing, and in particular to a distributed training method, device, device, and storage medium for models. Background technique [0002] At present, the existing methods for solving the optimal device segmentation on heterogeneous devices are mainly realized through the method of dynamic programming, and the dynamic programming method generally decomposes the problem into sub-problems for solution. However, the solution range of the dynamic programming method is limited. Once the modeling method is changed and model constraints are added, it is difficult to obtain the optimal value. Contents of the invention [0003] The present disclosure provides a model distributed training method, device, equipment and storage medium. [0004] According to an aspect of the present disclosure, a distributed training method o...

AI Technical Summary

Problems solved by technology

However, the solution range of the dynamic programming method is limited. Once the modeling method is changed and model constraints are added, it is difficult to obtain the optimal value

Images