Quantitative model deployment method and system, storage medium, and equipment

Abstract

The invention provides a quantitative model deployment method and system, a storage medium, and equipment. The method comprises the following steps: carrying out classification neural network retraining on a model through a quantitative perception training module, and obtaining a pseudo-quantitative model; reading the pseudo-quantization model into a deep learning compiling framework, wherein the deep learning compiling framework analyzes the pseudo-quantization model, identifies each node of the pseudo-quantization model, and performs convolution calculation to obtain a quantized four-bit model; compiling a rear end of the deep learning compiling framework, so as to enable the rear end to support the reasoning of the quantized four-bit model generated by the deep learning compiling framework; putting the quantized four-bit model into a model warehouse of a server, and creating a configuration file for calling the quantized four-bit model. According to the embodiment of the invention, the problem that model deployment is incompatible is solved; meanwhile, data transmission is reduced, and the data transmission bottleneck is avoided. In addition, the invention also relates to a method for executing reasoning on the basis of a quantitative model.

Description

technical field [0001] The present invention relates to the technical field of servers, in particular to a method, system, storage medium and equipment for quantitative model deployment. Background technique [0002] As early as 2016, Google launched TensorFlow Serving, a service-oriented framework for TensorFlow, which can expose the TensorFlow model to the outside world in the form of web services, accept request data from clients (Client) through network requests, calculate forward inference results and return. Triton's functionality is similar to TensorFlowServing. [0003] Model building and training are usually time-consuming and labor-intensive, and algorithm engineers need to do a lot of work to complete the building and training of a relatively complete model. The main purpose of the trained model is to solve practical problems more effectively, so deployment is a very important stage. Currently, however, the deployment of models is often problematic. For exampl...

AI Technical Summary

Problems solved by technology

[0007] In view of this, the object of the present invention is to propose a method, system, storage medium and equipment for deploying a trained model, so as to solve the problem of inference deployment and reasoning that the traditional reasoning framework in the prior art does not support the Dibit model. Slow speed, bottlenecks caused by data transmission, etc.

Images