Model dynamic training, checking, updating maintenance and utilization method under cloud platform

Abstract

The invention belongs to the technical field of machine learning, and discloses a model dynamic training, checking, updating maintenance and utilization method under a cloud platform. The resource manager obtains a workflow table according to different service requests and historical model training results; The model is verified by the verification data, and the result is notified to the resourcemanager; The service manager releases resources; And the resource manager re-issues the service to the scheduler of the service pool, and starts a new computing module for the service module. According to the invention, a lot of manual labeling cost is reduced; A large amount of model monitoring statistical data is obtained through the resource management module and used for solving the problem ofexploring and utilizing balance of the model monitoring statistical data and the original data, the model trained in the process and the original data are multiplexed to a certain extent, and after alarge amount of data is accumulated, a set of efficient workflow can be completed through excellent intelligent arrangement of the model monitoring statistical data. According to the method, hardwareresources are virtualized by utilizing the characteristics of a cloud platform, the characteristics of all functional modules are fully utilized, and the resources are utilized to the maximum extent.

Description

technical field [0001] The invention belongs to the technical field of machine learning, and in particular relates to methods for dynamic training, verification, update maintenance and utilization of models under a cloud platform. Background technique [0002] At present, the existing technologies commonly used in the industry are as follows: [0003] Machine learning refers to algorithms that use algorithms to analyze data patterns and use patterns to predict results. It is divided into supervised learning, unsupervised learning, and reinforcement learning. Deep learning is an extension of the neural network algorithm in machine learning. It is the second stage of machine learning - deep learning (the first stage is shallow learning), where depth refers to the number of layers of the neural network. Deep learning is essentially a process of hierarchical feature extraction and learning. It builds a multi-layer hidden neural network model, uses massive data to train model fe...

AI Technical Summary

Problems solved by technology

The original training data and training models in different stages are not fully utilized, resulting in a certain degree of waste

[0005] 2) The training mode needs to be optimized

At present, the training of deep learning models is still dominated by supervised learning in most cases, and the development of supervised learning requires a large number of manually labeled training samples. This process often takes a long time and requires a lot of manpower, which is extremely complicated.

[0006] 3) It is difficult to balance between exploration and utilization

[0007] 4) Lack of efficient workflow

[0009] (1) The data utilization rate of the existing technology is not high; the existing deep learning model training is still based on supervised learning in most cases, and the development of supervised learning requires a large number of manually labeled training samples, and this process often consumes For a long time, it also requires a lot of manpower and is extremely complicated, so the training mode needs to be optimized

[0010] (2) In the actual application field, in the big data environment, the number of parameters to be learned is large, and it is difficult to balance optimization exploration and utilization; and there is a lack of efficient workflow

[0013] The technical difficulty lies in how to complete the dynamic training, verification, update maintenance and utilization of the model under the cloud platform with an efficient workflow on the basis of improving data utilization and reducing labor and time costs

Images