Novel multi-algorithm access method and server

Abstract

The invention relates to the field of algorithm deployment, and discloses a novel multi-algorithm access method and a server. The method comprises the following steps that: an algorithm service is configured, a configuration file is compiled, and the algorithm service is configured according to algorithm service configuration information; the algorithm service is started, the algorithm service is started through the monitoring process, and the monitoring process is started according to configuration of the algorithm service; the algorithm service applies for connection, and the algorithm service applies for connection according to the configured algorithm service; and an algorithm is called, and according to the input of the algorithm, an algorithm request is submitted to the algorithm service so as to call the algorithm. On an algorithm server with a GPU, the application of the algorithm service is used, the used resources of CPU / GPU copy bandwidth and video memory are less, the utilization rate of the GPU is higher, more algorithm requests can be accessed, and the use efficiency of the server is further improved.

Description

technical field [0001] The present invention relates to the field of algorithm deployment, in particular to a new method and server for accessing multi-algorithms based on the deployment of conventional algorithms and deep learning algorithms on servers. Background technique [0002] With the rapid development of deep learning in the past few years, the number of products in computer vision and data processing has increased day by day, and the corresponding algorithm server also needs to support the simultaneous operation of multiple algorithms and access to multi-channel front-end devices. The current way of server management algorithm has the following problems: [0003] If each user application independently accesses the algorithm backend, each application will independently apply for algorithm resources. If these applications have parameters that use the same algorithm (such as the model parameters of the deep learning algorithm), resources will be wasted; at the system...

AI Technical Summary

Problems solved by technology

If these applications have parameters that use the same algorithm (such as the model parameters of the deep learning algorithm), resources will be wasted; at the system level, the priority of the algorithm and the competition for resources cannot be controlled, resulting in unexpected results

[0004] Current open source algorithm services (such as Nvidia's Triton) cannot realize the serialization of different algorithms in a single request, resulting in unnecessary data copy and synchronization; The service does not provide a specific optimization solution, and the resources are not fully scheduled to meet the demand

[0005] The current open source algorithm services mainly provide access to the backend of deep learning algorithm models. They are not friendly to the access support of non-deep learning algorithms, and the support for algorithm pre-processing and algorithm post-processing cannot meet the needs.

Images