Low-latency machine learning as service generation method

Abstract

The invention discloses a low-latency machine learning as a service generation method. A model abstraction layer provides a universal API, and the heterogeneity of a machine learning framework and a model can be abstracted; the model parameter optimization layer is located above the model abstraction layer and is responsible for dynamically selecting, combining and optimizing parameters of the prediction model; the model management layer manages the model through the information of the model storage layer, so that the application end can obtain the optimal prediction service dynamically without perception; the event service layer is responsible for interacting with an application end, storing information requested by a user of the application end and feeding back the information to the model layer so as to dynamically and automatically optimize model parameters; and the model monitoring layer is responsible for calling and monitoring online model services so as to discover problems occurring in the calling process in time and improve the transparency of model operation and maintenance. The complexity of an existing prediction service stack is reduced, and key attributes such as lowdelay, high throughput and model accuracy of prediction services are achieved.

Description

technical field [0001] The invention relates to the application field of remote communication and machine learning methods, in particular to a low-latency machine learning-as-a-service generation method. Background technique [0002] With the informatization of society, the increasing popularity of mobile Internet, the accumulation of big data, the increasing maturity of computing power, and a large number of hot public opinion events on AlphaGo in the public media, artificial intelligence is on a journey to break the ice, and the global attitude towards artificial intelligence is also changing. Doubt, fear transforms into curiosity and active embrace. At present, various types of enterprises at home and abroad are carrying out or preparing for the transformation or application of artificial intelligence. In fact, artificial intelligence that appears in the media is mostly a concept, but when it is implemented, it will be embodied in machine learning, deep neural network, I...

AI Technical Summary

Problems solved by technology

Therefore, the deployment, model optimization, and maintenance management of machine learning services are quite difficult and error-prone. In addition, another disadvantage is that in order to complete a machine learning application, it usually requires three teams of algorithm engineers, development engineers, and operation and maintenance engineers. The development and deployment of the model can only be achieved through the cooperation of people, which is a great waste of manpower and efficiency, and the knowledge reserves of the three teams are also different, and the communication cost is also high, which also affects the promotion and management of the project. difficulty

Images