Preloading method of algorithm model in container cluster

Abstract

The embodiment of the invention provides a preloading method and device for an algorithm model in a container cluster and computing equipment, realizes preloading of the algorithm model in the container cluster, and solves the problems of slow real-time loading of the model and loading queue blockage and model loading failure caused by simultaneous loading of a large number of models in a short time. The method comprises the steps of obtaining algorithm model loading task distribution, idle model longest survival time and algorithm model loading speed in a first time period; determining algorithm model preloading task distribution in a second time period earlier than a first time period according to the algorithm model loading task distribution in the first time period, the longest survival time of the idle model and the algorithm model loading speed; and pre-loading the task distribution pre-loading algorithm model according to the algorithm model in the second time period.

Description

technical field [0001] The present invention relates to the technical field of containers, in particular to a method, device and computing device for preloading algorithm models in container clusters. Background technique [0002] Related applications of artificial intelligence need to load corresponding algorithm models when providing services. In the usage scenarios of intelligent voice interaction and intelligent customer service text interaction, because different customers have different business content, different algorithm models need to be provided to different customers. In the case of a large number of customers, loading the algorithm model will consume huge system resources, so in actual use, the algorithm model is designed to be loaded on demand, and then loaded through the model instance management system based on container technology when customers need to use it , if the model has not been used for a period of time, the model instance management system will un...

AI Technical Summary

Problems solved by technology

[0003] The existing technology solves the problem that a large number of model instances occupy system resources when they are idle, but it takes a certain amount of time to load the model with the container, and some artificial intelligence systems that require real-time response will therefore take a long time when the model is called for the first time. The delay affects the user experience

On the other hand, the usage time of most artificial intelligence systems is very consistent. For example, the intelligent outbound call system usually starts tasks intensively at 9:00 am and 2:00 pm, and the customer service system is also used intensively during morning working hours. The specificity of this user usage leads to a large number of algorithm models being loaded within a certain period of time, which can easily lead to blocking of the loading queue, making the model loading time longer and increasing the probability of model loading failure

Images