Model online deployment method and device

Abstract

The invention provides a model online deployment method and device, and the method comprises the steps: receiving training data, carrying out the training of a model based on TensorFlow according to the training data, and obtaining a trained model; storing the trained model based on a preset path, and packaging the model, the model having an alias and a version number corresponding to the model; generating a corresponding container based on the packaged model, wherein the container calls the packaged model based on a version number and an alias; and respectively deploying the trained model andthe container to an infrastructure of a Paas online platform based on the Kubernetes. Compared with the traditional mainstream mode, algorithm engineers do not need to pay attention to model serviceconstruction and can focus on researching the research itself; model updating and multi-model deployment can be rapid and efficient, short-time online operation and verification of more models becomepossible, scheduling management of the model service is more reliable through combination with Kubernetes, dynamic adaptation to page view change can be realized, and the service is more reliable.

Description

technical field [0001] The present invention relates to artificial intelligence technology, in particular to a method and device for online model deployment. Background technique [0002] AI service has become one of the most important reliance of Internet companies today, and it is the core technology for companies to maintain rapid growth and efficient operations. The existing service mode of AI is mainly based on the existing big data, through algorithms, training the model offline, and then using the online service method to serve the results generated by the model to the outside world. [0003] However, how to use the model trained in the offline environment for online deployment and achieve near real-time inference has always been a difficult point in the industry. The current mainstream methods mainly include the following: [0004] 1. Offline training + cache mode. Generate recommendation results for users in the offline environment every once in a while, and stor...

AI Technical Summary

Problems solved by technology

[0009] 1. Model update iteration is slow, and it takes a lot of time to write server code;

[0010] 2. There is no guarantee of hot update of the service, and the model update can be completed without interrupting the service;

[0011] 3. In the connection process between the offline training model and the online service, it is impossible to quickly and effectively deploy the offline training model to the online service;

[0012] For the above three problems, there is no technical solution that can solve them at the same time

Images