Modeling method and device for distributed machine learning, and equipment

Abstract

Disclosed are a modeling method and device for distributed machine learning, and equipment. By setting a corresponding target modeling unit for an acquired training data set, the target modeling unit is one of the following: a classification modeling unit, a regression modeling unit or a clustering modeling unit; a parameter selection mode and a verification mode are configured for each modeling algorithm included in the target modeling unit to obtain a plurality of initial modeling algorithms and a plurality of groups of training subsets and verification subsets; the training subsets in each group are respectively input into each initial modeling algorithm, and a prediction model of each initial modeling algorithm is obtained according to a distributed task scheduling strategy; and each prediction model is evaluated according to the evaluation parameters to obtain a target initial modeling algorithm meeting a preset condition. The target initial modeling algorithm is trained according to the training data set to obtain the target prediction model, and the to-be-predicted data is predicted through the target prediction model, so that the time of automatic modeling is shortened, the skill requirements on analysts are reduced, and machine learning is more intelligent.

Description

technical field [0001] The present invention relates to the technical field of automated machine learning, in particular to a modeling method, device and equipment for distributed machine learning. Background technique [0002] With the development of big data, machine learning applications have been gradually implemented in combination with the industry. However, in the process of machine learning research and application, machine learning algorithms need to be configured and optimized for each different real-world scenarios. Data analysts usually need to Putting a lot of time and energy into model tuning also increases the skill requirements for data analysts. If it is possible to automatically determine which model structures will produce better results, without the need to use manpower to try different algorithms and adjust parameters for the algorithms, it will undoubtedly improve the efficiency of research. Based on this idea, automated machine learning has gradually ...

AI Technical Summary

Problems solved by technology

At present, mainstream manufacturers have also proposed many mature automatic machine learning technologies, but they are all aimed at specific computing frameworks and are not suitable for current distributed machine learning.

Images