Machine learning tool middleware and machine learning training method

Abstract

The invention discloses machine learning tool middleware and a machine learning training method. A machine learning tool comprises at least one training unit, each training unit is provided with the middleware combined with the machine learning tool, and the middleware comprises a bottom layer communication module and at least one of a data distribution module, a model parameter updating module, a training parameter adjustment module and a training stop judging module. The training units distribute needed data from data storage equipment to storage units capable of being accessed by the training units, and therefore the training units can read the data from the storage units for training; training unit model parameters are updated and adjusted by means of communication of the corresponding modules among the middleweare; whether training stops or not is judged on the basis of training information of all the training units. The middleware is responsible for processing needed by large-scale parallel training, and therefore various machine learning tools are conveniently extended.

Description

technical field [0001] The invention belongs to the technical field of machine learning, and in particular relates to a machine learning tool middleware and a machine learning training method. Background technique [0002] Machine learning is a branch of artificial intelligence, and in many cases, it has almost become synonymous with artificial intelligence. Simply put, machine learning is to use machine learning algorithm models to enable machines to learn laws from a large amount of historical data, so as to intelligently identify new samples or make predictions for the future. The general process of machine learning is to calculate the machine learning algorithm model parameters from the input data (input data), form the machine algorithm model according to the calculated model parameters, and intelligently identify new samples or make predictions for the future. In many practical applications, the input data is very large and must be processed by multiple computing devi...

AI Technical Summary

Problems solved by technology

However, in doing so, a large number of experimental comparisons and verifications are required, and existing products need to be modified to be compatible with the data and model formats of the platform.

In addition, there is no guarantee that the existing implementation of the platform can meet the business requirements

At the same time, it is necessary to have an in-depth understanding of the implementation of the platform, and it takes a lot of time to realize the data format, model format and training method, which has high requirements for users

Images