Visual large model distributed training method and system

Abstract

The invention discloses a visual large model distributed training method and system, and the method comprises the steps: constructing a distributed training system which comprises a main control server, a plurality of GPU servers, a distributed storage server, and a storage network switch; determining a data loading mode according to the size of a data set used for visual large model training; according to structural characteristics of the visual large model, evaluating parameter quantities and calculation quantities of different types of network layer groups in the visual large model, decomposing the visual large model in combination with computing power and caching capability of a GPU, and determining a parallel training scheme of the visual large model; performing model training by adopting a hybrid parallel mode, performing model aggregation, global model updating and model distribution based on the master control server, evaluating the visual large model by utilizing a training set and a verification set every a plurality of cycles in the training process, and stopping training after the visual large model converges. According to the invention, efficient training of the visual large model is realized.

Description

technical field [0001] The invention relates to the field of computer application technology, in particular to a distributed training method and system for a large visual model. Background technique [0002] With the rapid development of computer vision, artificial intelligence, communication technology, processor hardware and other application technologies, video surveillance equipment, with its huge advantages of rich information, many types of supporting tasks, and low cost, is widely used in production, life, and urban management. are playing an increasingly important role. At present, the deployment and application of various video devices are becoming more and more extensive, and the amount of visual data has also increased dramatically. How to effectively and fully use and process visual big data is an important issue related to the development of various application fields. [0003] In the field of artificial intelligence-based vision applications, the size of label...

AI Technical Summary

Problems solved by technology

[0007] The main purpose of the present invention is to provide a large-scale visual model distributed training method and system, aiming to solve the problem in the prior art that it is difficult to efficiently train large-scale visual models by using visual big data due to the limitation of data transmission rate

Images