Model calling method and apparatus, and device, storage medium and program product

Abstract

Provided in the present disclosure are a model calling method and apparatus, and a device, a storage medium and a program product. The method comprises: acquiring a model calling request, wherein the model calling request is used for requesting the calling of a target model for task processing; if the state of the target model is a loaded state, calling the target model to perform task processing, wherein the state of the target model is dynamically updated on the basis of historical calling information for the target model, and the loaded state is used for indicating that the target model has been loaded into a container; and if the state of the target model is an unloaded state, loading the target model, and calling the target model to perform task processing, wherein the unloaded state is used for indicating that the target model has not been loaded into the container. By means of the model calling method in the present disclosure, any model can be called, thereby implementing the calling of a large number of models.

Description

[0001] Model Invocation Method, Apparatus, Device, Storage Medium, and Program Product This disclosure claims priority to Chinese Patent Application No. 202410684544.8, filed on May 29, 2024, entitled "Model Invocation Method, Apparatus, Device, Storage Medium, and Program Product," the entire contents of which are incorporated herein by reference. Technical Field This disclosure relates to the field of computers, and more particularly to a model invocation method, apparatus, device, storage medium, and program product. Background Art Machine learning models (hereinafter referred to as "models") can be used to process prediction tasks. For example, personalized products can be recommended to users through models. In related technologies, different models can be built for different types of tasks on a single platform, resulting in multiple models. For each type of task, the corresponding model can be used to process that type of task, improving the processing accuracy of the task. However, in practice, each model is used, but due to limited server resources, it is impossible to deploy a large number of models simultaneously. Therefore, how to invoke a large number of models is a problem that urgently needs to be solved. The present disclosure provides a model invocation method, apparatus, device, storage medium, and program product to solve the problem of invoking a large number of models. In a first aspect, embodiments of the present disclosure provide a model invocation method, comprising: obtaining a model invocation request, the model invocation request being used to request the invocation of a target model for task processing; if the target model is in a loaded state, then invoking the target model for task processing, the state of the target model being dynamically updated based on historical invocation information of the target model, the loaded state indicating that the target model has been loaded into a container; if the target model is in an unloaded state, then loading the target model and invoking the target model for task processing, the unloaded state indicating that the target model has not been loaded into a container. In one possible implementation, loading the target model includes: obtaining a configuration file and a model file of the target model; determining a target container, the target container being used to load the target model; and loading the target model into the target container according to the configuration file and the model file. In one possible implementation, obtaining the configuration file and model file of the target model includes: if the model file exists in a model cache, then obtaining the model file from the model cache; if the model file does not exist in the model cache, then obtaining the model file from a model repository, wherein the model repository is used to store model files; and generating the configuration file based on the model file.In one possible implementation, obtaining the model file from the model repository includes: determining the storage information of the target model, the storage information including the storage location and storage type of the target model in the model repository; determining a caching engine based on the storage type, the caching engine being used to obtain the model file corresponding to the storage type; and obtaining the model file from the model repository through the caching engine. In one possible implementation, determining the target container includes: determining at least one computing node corresponding to the target model, wherein the computing node includes at least one processing instance and at least one container corresponding to each processing instance; determining the remaining capacity of each container in the at least one computing node and the target capacity required by the target model; and determining the target container O based on the remaining capacity of each container in the at least one computing node and the target capacity. In another possible implementation, determining the target container based on the remaining capacity of each container in the at least one computing node and the target capacity includes: determining, based on the remaining capacity of each container in the at least one computing node and the target capacity, whether there is at least one available container in the at least one computing node, wherein the remaining capacity of the available container is greater than or equal to the target capacity; if yes, determining the target container among the at least one available container; if no, determining the target computing node among the at least one computing node and creating the target container in the target computing node. In one possible implementation, determining a target computing node among the at least one computing node includes: acquiring historical model execution information for each computing node, the historical model execution information including the identifier of the model run by the computing node within a historical time period and the model execution time; determining the target computing node among the at least one computing node based on the historical model execution information for each computing node, wherein the target computing node has run the target model within the historical time period, and the time interval between the model execution time of the target model and the current time is the minimum. In another possible implementation, loading the target model into the target container based on the configuration file and the model file includes: storing the model file in a public storage space; configuring the model file in the public storage space according to the configuration file and the target container to load the target model into the target container.In one possible implementation, the method further includes: determining the unused duration of each model in a loaded state in a computing node; determining a model to be uninstalled in the computing node based on the unused duration of each model, wherein the unused duration of the model to be uninstalled is greater than or equal to a preset duration; and uninstalling the model to be uninstalled in the computing node. In a second aspect, embodiments of this disclosure provide a model invocation device, the device comprising: an acquisition module, an invocation module, and a loading module, wherein the acquisition module is configured to: acquire a model invocation request, the model invocation request being used to request the invocation of a target model for task processing; the invocation module is configured to: if the target model is in a loaded state, invoke the target model for task processing, the state of the target model being dynamically updated based on historical invocation information of the target model, the loaded state being used to indicate that the target model has been loaded into a container; the loading module is configured to: if the target model is in an unloaded state, load the target model, the unloaded state being used to indicate that the target model has not been loaded into a container; the invocation module is further configured to: invoke the target model for task processing. In one possible implementation, the loading module is specifically used to: obtain the configuration file and model file of the target model; determine a target container, the target container being used to load the target model; and load the target model in the target container according to the configuration file and the model file. In another possible implementation, the loading module is specifically used to: if the model file exists in a model cache, obtain the model file from the model cache; if the model file does not exist in the model cache, obtain the model file from a model repository, the model repository being used to store the model file; and generate the configuration file according to the model file. In another possible implementation, the loading module is specifically used to: determine the storage information of the target model, the storage information including the storage location and storage type of the target model in the model repository; determine a caching engine according to the storage type, the caching engine being used to obtain the model file corresponding to the storage type; and obtain the model file from the model repository through the caching engine.In one possible implementation, the loading module is specifically configured to: determine at least one computing node corresponding to the target model, wherein the computing node includes at least one processing instance and at least one container corresponding to each processing instance; determine the remaining capacity of each container in the at least one computing node and the target capacity required by the target model; and determine the target container based on the remaining capacity of each container in the at least one computing node and the target capacity. In another possible implementation, the loading module is specifically configured to: determine whether there is at least one available container in the at least one computing node, wherein the remaining capacity of the available container is greater than or equal to the target capacity, based on the remaining capacity of each container in the at least one computing node and the target capacity; if yes, determine the target container among the at least one available container; if no, determine the target computing node in the at least one computing node and create the target container in the target computing node. In one possible implementation, the loading module is specifically used to: acquire historical model running information for each computing node, the historical model running information including the identifier of the model run by the computing node within a historical time period and the model running time; and, based on the historical model running information of each computing node, determine the target computing node among the at least one computing node, the target computing node having run the target model within the historical time period, and the time interval between the model running time of the target model and the current time being the minimum. In another possible implementation, the loading module is specifically used to: store the model file in a public storage space; and, based on the configuration file and the target container, configure the model file in the public storage space to load the target model into the target container. In one possible implementation, the apparatus further includes: a first determining module, a second determining module, and an unloading module, wherein the first determining module is configured to determine the unused duration of each model in the loaded state in the computing node; the second determining module is configured to determine a model to be unloaded in the computing node based on the unused duration of each model, wherein the unused duration of the model to be unloaded is greater than or equal to a preset duration; and the unloading module is configured to unload the model to be unloaded in the computing node. In a third aspect, embodiments of this disclosure provide an electronic device, including: a memory and a processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, causing the processor to perform the method described in any one of the first aspects.Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method described in any of the first aspects. Fifthly, embodiments of this disclosure provide a computer program product, including a computer program, which, when executed by a processor, implements the method shown in any of the first aspects. Embodiments of this disclosure provide a model invocation method, apparatus, device, storage medium, and program product, allowing an electronic device to obtain a model invocation request. If the target model is in a loaded state, the electronic device can invoke the target model for task processing; if the target model is in an unloaded state, the electronic device can load the target model and invoke it for task processing. When a target model needs to be used, it can be invoked using the model invocation method of this disclosure. The target model can be any one of a large number of models, and the electronic device can invoke any one model, thereby enabling the invocation of a large number of models. The accompanying drawings, which are included to provide a further understanding of this disclosure and constitute a part of this disclosure, illustrate exemplary embodiments of this disclosure and are used to explain this disclosure, but do not constitute an improper limitation of this disclosure. In the accompanying drawings: Figure 1 is a schematic diagram of a scenario provided by an exemplary embodiment of the present disclosure; Figure 2 is a flowchart of a model invocation method provided by an exemplary embodiment of the present disclosure; Figure 3 is a flowchart of another model invocation method provided by an exemplary embodiment of the present disclosure; Figure 4 is a schematic diagram of at least one computing node corresponding to a target model provided by an exemplary embodiment of the present disclosure; Figure 5 is a flowchart of yet another model invocation method provided by an exemplary embodiment of the present disclosure; Figure 6 is a structural schematic diagram of a model invocation system provided by an exemplary embodiment of the present disclosure; Figure 7 is a structural schematic diagram of a model invocation device provided by an exemplary embodiment of the present disclosure; Figure 8 is a structural schematic diagram of another model invocation device provided by an exemplary embodiment of the present disclosure; Figure 9 is a structural schematic diagram of an electronic device provided by an exemplary embodiment of the present disclosure. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of related data must comply with relevant laws, regulations, and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse. To make the objectives, technical solutions, and advantages of this disclosure clearer, the technical solutions of this disclosure will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings.Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure. Figure 1 is a schematic diagram of a scenario provided by an exemplary embodiment of this disclosure. Referring to Figure 1, if there are multiple models, these multiple models can be Model 1, Model 2, ..., Model n. These n models can be models in the same platform. The function of each model can be different. For example, if n is 4, there are 4 models. Assuming that these 4 models are models in the same shopping platform, then Model 1 can be used for fraud detection, Model 2 can be used to recommend personalized products to users, Model 3 can be used for intelligent search ranking, and Model 4 can be used for inventory demand forecasting. The electronic device can obtain a model call request, determine the target model among the multiple models, and then call the target model to process the model call request. For example, if n is 4, meaning there are 4 models, the electronic device can determine the target model as model 2 from among the 4 models based on the model call request, and can call model 2 to process the model call request, i.e., recommend personalized products to the user. In related technologies, different models can be built for different types of tasks on a single platform, resulting in multiple models. For each type of task, the corresponding model can be used to process that type of task, improving the processing accuracy. However, in actual work, each model is used, but due to limited server resources, it is impossible to deploy a large number of models simultaneously. Therefore, how to call a large number of models is a problem that urgently needs to be solved. To solve the above technical problem, this disclosure provides a model calling method. In this method, after the electronic device obtains the model call request, it can determine the state of the target model and call the target model for task processing based on the state of the target model. Through the above method, it is not necessary to deploy a large number of models simultaneously; the electronic device can call any one model, thereby realizing the calling of a large number of models. The technical solution shown in this disclosure will be described in detail below through specific embodiments. It should be noted that the following embodiments can exist independently or in combination. For identical or similar content, the description will not be repeated in different embodiments. Figure 2 is a flowchart illustrating a model invocation method provided by an exemplary embodiment of this disclosure. Referring to Figure 2, the method may include:

[0002] 5201. Obtaining a Model Invocation Request. The execution entity in this embodiment can be an electronic device or a model invocation device installed in the electronic device. The model invocation device can be implemented in software or a combination of software and hardware. The model invocation device can be a processor in the electronic device. For ease of understanding, the following description uses an electronic device as the execution entity. A model invocation request is used to request the invocation of a target model for task processing. The model invocation request may include the identifier of the target model and the request content. For example, model invocation request 1 can be used to request model 2 to recommend personalized products to user A. The model invocation request may include the identifier of model 2 as "2" and the request content as recommending personalized products to user A. A client can send a model invocation request to the electronic device so that the electronic device can obtain the model invocation request. For example, if the client can send model invocation request 1 to the electronic device, the electronic device can obtain model invocation request 1, which can be used to request model 2 to predict and recommend personalized products to user A.

[0003] 5202. If the target model's state is "loaded," then the target model is invoked for task processing. Since the model invocation request includes the target model's identifier, after the electronic device obtains the model invocation request, it can determine the target model based on the identifier and its state. The target model's state can include a loaded state and an unloaded state. A loaded state can indicate that the target model has been loaded into the container; an unloaded state can indicate that the target model has not been loaded into the container. The target model's state can be dynamically updated based on historical invocation information. In an optional embodiment, the historical invocation information can include the last invocation time of the target model. If the time difference between the historical invocation time and the current time is less than a preset duration, it indicates that the target model has not yet been unloaded, and the target model's state can be determined to be "loaded." If the time difference between the historical invocation time and the current time is greater than or equal to the preset duration, the target model can be unloaded, and its state updated to "unloaded." It should be noted that if the historical call information does not contain a previous call time for the target model, it means that the target model has not been loaded, and the target model's state can be determined to be unloaded. In another optional embodiment, the historical call information may include at least one historical state corresponding to the target model, and the state update time corresponding to each historical state. The state update time corresponding to each historical state may include the latest state update time. If the historical state corresponding to the latest state update time is a loaded state, the target model's state can be determined to be loaded; if the historical state corresponding to the latest state update time is an unloaded state, the target model's state can be determined to be unloaded. For example, if the target model is model 2, and if model 2's state is loaded, and if model call request 1 is used to request model 2 to predict and recommend personalized products to user A, then the electronic device can call model 2 to process model call request 1, that is, recommend personalized products to user A through model 2.

[0004] S203. If the target model is in an unloaded state, then load the target model and call it for task processing. In an optional embodiment, the target model can be loaded as follows: obtain the target model's configuration file and model file; determine the target container; load the target model into the target container according to the configuration file and model file. The configuration file may include the target model's model identifier, model format, and parameters required when loading the model. The configuration file can be used to load and run the model. For example, configuration file 2 can be as follows:

[0005] 1{

[0006] 2 "name": "model-2",

[0007] 3 "implementation" : "mlserver_sklearn.SKLeamModel",

[0008] 4 "parameters": {

[0009] 5 "uri": ". / model-a.pkl",

[0010] 6 "version": "1"

[0011] 7}

[0012] 8} Wherein, "name" is used to define the identifier of the model; "implementation" is used to define the type of computation node used to load the model; "parameters" is used to define the parameters required when loading the model, usually including the model's storage location (URI) and version. The model file can include model structure information and model parameters, etc. The model file can be stored in a specific file directory to ensure that it can be correctly identified. For example, the specific file directory can be as follows:

[0013] / models

[0014] / model- 1

[0015] / I

[0016] / model-specific-files model- settings.] son

[0017] / model-2

[0018] / I

[0019] In this file directory, " / models" represents the root directory containing all models; " / model-1" and " / model-2" represent the directories of specific models, serving as model identifiers; " / 1" represents a specific version of the model, a numeric directory representing the version number; " / model-specific-files" represents the directory of model files; and "model-settings.json" represents the model's configuration file. A target container can be used to load and run the target model. For example, the target container can be container 1. For example, if the target model is model 2, and model 2 is in an unloaded state, the electronic device can obtain the configuration file 2 and model file 2 of model 2, and determine the target container. Assuming the target container determined by the electronic device is container 1, the electronic device can load model 2 into container 1 based on the configuration file 2 and model file 2. In another optional embodiment, the target model can be loaded in the following way: obtaining the configuration file and model file of the target model; determining the target computing node; and loading the target model into the target computing node based on the configuration file and model file. The target computing node can be the computing node that has run the target model within a historical time period, and whose time interval between the running time of the target model and the current time is the shortest. Optionally, the target computing node can be a physical server or a virtual machine. For example, if the target model is model 2 and the state of model 2 is unloaded, the electronic device can obtain the configuration file 2 and model file 2 of model 2 and determine the target computing node. Assuming that the target computing node determined by the electronic device is computing node 1, the electronic device can load model 2 in computing node 1 according to the configuration file 2 and model file 2. After loading the target model, the electronic device calls the target model to process the model call request. For example, if the target model is model 2, and model call request 1 is used to request the call of model 2 to predict and recommend personalized products to user A, then after loading model 2, the electronic device can call model 2 to process model call request 1, that is, recommend personalized products to user A through model 2. It should be noted that after the electronic device loads the target model, it can update the state of the target model from unloaded to loaded. In this embodiment of the disclosure, the electronic device can obtain the model call request.If the target model is in a loaded state, the electronic device can call the target model for task processing; if the target model is in an unloaded state, the electronic device can load the target model and call it for task processing. Using this method, it is not necessary to deploy a large number of models simultaneously. When the target model needs to be used, it can be called using the model calling method of this disclosure. The target model can be any one of a large number of models, and the electronic device can call any one model, thereby achieving the calling of a large number of models. Below, based on the embodiment shown in Figure 2 and in conjunction with Figure 3, the above model calling method will be described in detail. Figure 3 is a flowchart illustrating another model calling method provided by an exemplary embodiment of this disclosure. Referring to Figure 3, the method may include:

[0020] 5301. Obtaining Model Invocation Requests. For example, if a client can send Model Invocation Request 1 to an electronic device, the electronic device can obtain Model Invocation Request 1. Model Invocation Request 1 can be used to request the invocation of Model 2 to predict and recommend personalized products to user A.

[0021] 5302. Determine the state of the target model. The electronic device can determine the state of the target model. If the target model is in an unloaded state, steps S303 to S311 can be executed; if the target model is in a loaded state, step S311 can be executed. For example, if the target model is model 2, the electronic device can determine the state of model 2. If model 2 is in an unloaded state, step S303 can be executed; if model 2 is in a loaded state, step S311 can be executed. 5303. Determine if a model file exists in the model cache. The model cache can be used to cache model files. The electronic device can determine if the model file of the target model exists in the model cache. If yes, step S304 can be executed; if no, step S305 can be executed. For example, if the target model is model 2, the electronic device can determine if model file 2 of model 2 exists in the model cache. If yes, step S304 can be executed; if no, step S305 can be executed.

[0022] 5304. Obtain the model file from the model cache. Since the model cache contains the model file of the target model, the electronic device can obtain the model file from the model cache. For example, if the target model is model 2, and the model cache contains model file 2 of model 2, then the electronic device can obtain model file 2 from the model cache.

[0023] 5305. Obtain model files from a model repository. A model repository can be used to store model files, code, and related resources for multiple models. The model repository can be local or cloud-based. It can be a central repository or system for storing and managing machine learning models. The model repository can provide features such as model version control, metadata management, and access control. Optionally, the model repository can support multiple data sources with different protocols, and multiple data sources can be mounted to different subdirectories under a unified namespace. Different data sources correspond to different models, resulting in different model storage types. In an optional embodiment, model files can be obtained from the model repository by: determining the storage information of the target model; determining the caching engine based on the storage type; and obtaining the model files from the model repository through the caching engine. The storage information may include the storage location of the target model in the model repository and the model's storage type. The model's storage type is also the storage type of the model's model files. Optionally, the electronic device can determine the storage information of the target model from the target model file resource object. The target model file resource object refers to the model file resource object corresponding to the target model. Model file resource objects can be represented and managed as custom resources within the Kubernetes container orchestration engine. Custom resources are a mechanism that extends native Kubernetes resources, allowing the definition and management of custom resource types within Kubernetes. Model file resource objects can be defined using custom resource definitions (CRDs), and can be created, updated, and deleted declaratively. Model file resource objects can be used to define the structure and behavior of model file resources, including the following functions:

[0024] (1) Used for data location: The model file resource object can include the storage location of the model in the model repository and the storage type. Therefore, the model file resource object can be used for data location to determine the physical storage location of the model.

[0025] (2) Data summary for storing model files: The model file resource object can store metadata about the model file (e.g., data size, number of files, etc.), which helps with data management and optimization. (3) Access optimization: Data caching and prefetching strategies can be defined in the model file resource object. For example, on which cache nodes should the model file data be cached, and how to dynamically adjust the cache size to adapt to the access pattern. Through the model file resource object, users do not need to care about the physical storage details of the model file, and electronic devices can efficiently access the model file. For example, if the target model is model 2, the electronic device can determine the model file resource object 2 corresponding to model 2 based on the model identifier "2" of the target model, and determine the storage information of model 2 in the model file resource object 2. Assuming that the storage information of model 2 can be determined, it includes the storage location of the target model in the model repository as storage location 1, and the storage type as optional. Multiple caching engines can exist. Different caching engines are used to obtain model files of different storage types. For any caching engine, the caching engine can have a corresponding caching engine resource object. The cache engine resource object defines a new resource type, which can include the following functionalities:

[0026] (1) Cache management: Intelligently cache data to nearby computing nodes according to access patterns to reduce data access latency and improve the performance of accessing the data.

[0027] (2) Data prefetching: Based on the predicted access pattern, the model file is loaded into the model cache in advance to reduce the waiting time for the first access.

[0028] (3) Elastic expansion and reclamation: Automatically adjust the size and allocation of cached resources in the model cache to adapt to changes in model load and optimize resource utilization efficiency.

[0029] (4) Data Summary and Monitoring: Collect and report information on data access and caching status to help users monitor and optimize model management strategies. After obtaining storage information, the electronic device can determine the corresponding caching engine based on the storage type, and then call the caching engine to retrieve the model file of that storage type from the model repository based on the corresponding caching engine resource object. For example, if the storage information includes the target model's storage location in the model repository as storage location 1 and the storage type as storage type 1, the electronic device can determine the corresponding caching engine as caching engine 1 based on storage type 1. Stored in the model cache for later use.

[0030] 5306. Generate a configuration file based on the model file. Optionally, after the electronic device obtains the model file of the target model, it can generate the configuration file of the target model through the model loading and unloading adapter. For example, if the target model is model 2, after the electronic device obtains model file 2 of model 2, it can generate configuration file 2 of model 2 through the model loading and unloading adapter.

[0031] 5307. Identify at least one compute node corresponding to the target model. The compute node can be a physical server or a virtual machine. A compute node can load and manage multiple models simultaneously to handle model call requests for multiple different models, improving resource utilization. The electronic device can identify at least one compute node corresponding to the target model. This at least one compute node can correspond to the same multi-model service runtime. A multi-model service runtime can support multiple types of models; therefore, this at least one compute node can support multiple types of models, including the target model. A multi-model service runtime refers to a runtime environment capable of simultaneously hosting and providing inference functions for multiple machine learning models. It should be noted that all at least one compute node corresponding to the target model can support running the target model, but not all of them have necessarily run the target model. A compute node can include at least one processing instance and at least one container corresponding to each processing instance. Optionally, the processing instance can be a Graphics Processing Unit (GPU) or a Central Processing Unit (CPU). If the compute node is a physical server, the processing instance can be any of the following: physical GPU, virtual GPU, GPU core, physical CPU, virtual CPU, or CPU core; if the compute node is a virtual machine, the processing instance can be a virtual GPU or a virtual CPU. For any processing instance, a container group (pod) can be provided on the processing instance, and a container group can include at least one container. The container can be used to run the model. The following description, with reference to Figure 4, illustrates at least one compute node corresponding to the target model. Figure 4 is a schematic diagram of at least one compute node corresponding to the target model provided by an exemplary embodiment of this disclosure. Referring to Figure 4, if the target model is model 2, the electronic device can determine at least one compute node corresponding to model 2.Assume the electronic device can determine that model 2 corresponds to 10 computing nodes, namely computing node 1, computing node 2, ..., computing node 10. These 10 computing nodes can correspond to a multi-model service runtime, i.e., multi-model service runtime lo. Computing node 1 can include processing instance 1 and processing instance 2, and processing instance 1 can have container 1 and container 2. Processing instance 2 can have container 3. Computing node 2 can include processing instance 3, and processing instance 3 can have container 4. Computing node 10 can include processing instance 14 and processing instance 15, and processing instance 14 can have container 20. Processing instance 15 can have container 21.

[0032] S3O8. Determine the remaining capacity of each container in at least one computing node, and the target capacity required by the target model. Since each computing node can include at least one processing instance and at least one container corresponding to each processing instance, and each container has a corresponding capacity, the electronic device can determine the remaining capacity of each container in at least one computing node. For example, if the target model is model 2, the electronic device can determine the target capacity corresponding to model 2. Assume the target capacity is 1G; if model 2 corresponds to 10 computing nodes, namely computing node 1, computing node 2, ..., computing node 10, assume the electronic device can determine the remaining capacity of each container in these 10 computing nodes as shown in Table 1: Table 1

[0033] S309. Determine the target container based on the remaining capacity and target capacity of each container in at least one computing node. In an optional embodiment, the target container can be determined based on the remaining capacity and target capacity of each container in at least one computing node as follows: Based on the remaining capacity and target capacity of each container in at least one computing node, determine whether there is at least one available container in at least one computing node; if yes, determine the target container among the at least one available container; if not, determine the target computing node among at least one computing node, and create the target container in the target computing node. The remaining capacity of the available container can be greater than or equal to the target capacity. For example, if the remaining capacity of each container in the 10 computing nodes corresponding to Model 2 is as shown in Table 1, and if the target capacity of Model 2 is 1G, then three available containers can be identified among the multiple containers in these 10 computing nodes: Container 1, Container 3, and Container 21. Optionally, after the electronic device identifies at least one available container, it can determine the target container among the at least one available container in the following way: Obtain the container-related information of each available container; Based on the container-related information of each available container, determine the target container among the at least one available container. The container-related information may include the identifier of the model running in the container during the historical period, the model running time, the supported model format, the supported compatibility protocols, etc. The target container can be the container that has run the target model during the historical period and has the shortest duration between the model running time of the target model and the current time. For example, if there are three available containers, namely Container 1, Container 3, and Container 21, and if the container-related information of these three available containers is as shown in Table 2: Table 2 The electronic device can determine, based on the container information in Table 2, that both container 1 and container 3 have run model 2. If the current time is 11:00 on 2024-05-06, and the model running time of model 2 in container 1 is 10:00 on 2024-04-21, and the model running time of model 2 in container 3 is 14:30 on 2024-05-04, it can be determined that the time between the model running time of model 2 in container 3 and the current time is the shortest. Therefore, container 3 is determined to be the container that most recently ran model 2, and thus can be identified as the target container. If at least one available container does not exist in at least one computing node, the electronic device can determine the target computing node in at least one computing node and create the target container in the target computing node. In an optional embodiment, the target computing node can be determined in at least one computing node by: obtaining the historical model running information of each computing node; and determining the target computing node in at least one computing node based on the historical model running information of each computing node. Historical model execution information can include the identifier of the model that the computing node ran within the historical time period, as well as the model execution time. The target computing node can be the computing node that has run the target model within the historical time period, and whose execution time between the target model's execution time and the current time is the shortest. For example, if the target model is model 2, and model 2 corresponds to 10 computing nodes, namely computing node 1, computing node 2, ..., computing node 10, then the electronic device can obtain the historical model execution information of these 10 computing nodes. Assume that the historical model execution information of these 10 computing nodes can be as shown in Table 3: Table 3 The electronic device can determine that both computing node 1 and computing node 2 have run model 2 based on the historical model running information shown in Table 3. If the current time is 11:00 on 2024-05-06, and the model running time of model 2 in computing node 1 is 14:30 on 2024-05-04, and the model running time of model 2 in computing node 2 is 11:35 on 2024-04-29, it can be determined that the time interval between the model running time of model 2 in computing node 1 and the current time is the shortest. Therefore, computing node 1 can be determined as the computing node that most recently ran model 2, and thus can be identified as the target computing node. The electronic device can create container 22 in computing node 1. If the capacity of container 22 is greater than the target capacity of model 2, then container 4 is the target container. 5310. Load the target model into the target container according to the configuration file and model file. In one optional embodiment, the target model can be loaded into the target container according to the configuration file and the model file as follows: The model file is stored in a public storage space; the model file in the public storage space is configured according to the configuration file and the target container to load the target model into the target container. Optionally, there may be at least one public storage space. This at least one public storage space may be independent of the compute node. The electronic device can store the model file in the public storage space closest to the compute node where the target container is located through a model loading and unloading adapter, so that the model file can be quickly obtained when loading the model into the target container. Optionally, the electronic device can configure the model file in the public storage space according to the configuration file and the target container through a model distribution controller to load the target model into the target container. For example, if the target model is model 2 and the target container is container 3, the electronic device can store model file 2 of model 2 in the public storage space, and configure model file 2 according to the configuration file 2 of model 2 and container 3 through the model distribution controller to load model 2 into container 3.

[0034] 5311. Call the target model for task processing. Optionally, the electronic device may have a metadata-aware dynamic traffic routing mechanism. Through this mechanism, the electronic device can correctly route model call requests to containers that have loaded the target model. The metadata may include historical model execution information for each computing node and container-related information for each container. The metadata can be dynamically updated as each model is dynamically loaded and unloaded. The metadata-aware traffic routing mechanism may include dynamic routing strategies. These strategies can be flexibly generated based on metadata. A dynamic routing strategy is a strategy that dynamically determines the routing of model call requests based on their characteristics and routing rules. Dynamic routing strategies allow model call requests to be dynamically routed to different models based on their attributes, tags, or other conditions. Optionally, dynamic routing strategies may include: routing model call requests to the target model, distributing traffic proportionally to different target models, or performing traffic control based on other rules. Optionally, there are two scenarios for calling the target model for task processing: Scenario 1: After executing step S302, step S311 is executed. In this scenario, the target model is initially in a loaded state. Multiple containers may be running the target model. The electronic device can determine the first container based on the dynamic routing strategy in the metadata-aware traffic routing mechanism and send a model call request to the first container to call the target model in the first container for task processing. For example, if the target model is model 2, and both container 1 and container 3 are running model 2, the electronic device can send model call request 1 to container 1 according to the dynamic routing strategy in the metadata-aware traffic routing mechanism, so that model 2 in container 1 can process model call request 1. Scenario 2: After executing step S310, step S311 is executed. In this scenario, only the target container is running the target model. The electronic device can call the target model to process the model call request according to the dynamic routing strategy in the metadata-aware traffic routing mechanism to perform task processing. For example, if the target model is model 2, and model call request 1 is used to request model 2 to recommend personalized products to user A, and the target container is container 22, then the electronic device can send model call request 1 to container 22 according to the dynamic routing strategy in the metadata-based dynamic traffic routing mechanism, so as to call model 2 to process model call request 1, that is, recommend personalized products to user A through model 2.In this embodiment, the electronic device can obtain a model invocation request and determine the status of the target model. If the target model is in a loaded state, the electronic device can invoke the target model for task processing; if the target model is in an unloaded state, the electronic device can determine whether a model file exists in the model cache. If so, the model file can be obtained from the model cache; otherwise, the model file can be obtained from the model repository. The electronic device can generate a configuration file based on the model file. The electronic device can determine at least one computing node corresponding to the target model, and determine the remaining capacity of each container in the at least one computing node, as well as the target capacity required by the target model. Then, based on the remaining capacity and target capacity of each container in the at least one computing node, the target container can be determined. The electronic device can load the target model in the target container based on the configuration file and the model file, and invoke the target model for task processing. Through the above method, it is not necessary to deploy a large number of models simultaneously. The electronic device can invoke any model, thereby realizing the invocation of a large number of models; and it can improve the resource utilization of each computing node. In the embodiments shown in Figures 2 and 3, the process of loading and calling the target model for task processing is mainly described. The process of unloading the model will be explained below with reference to Figure 5. Figure 5 is a flowchart illustrating another model calling method provided by an exemplary embodiment of this disclosure. Referring to Figure 5, the method may include:

[0035] S501. Determine the unused duration of each model in the loaded state in the computing node. The unused duration refers to the time between the last time the model was called and the current time. For any computing node, the electronic device can determine each model in the loaded state in the computing node and obtain the unused duration of each model. For example, for computing node 1, if the models in the loaded state in the computing node are model 1, model 2, and model 3, the electronic device can obtain the unused duration of these three models. Assume that the unused duration of these three models is as shown in Table 4: Table 4

[0036] S502. Based on the unused duration of each model, determine the model to be unloaded in the computing node. The unused duration of the model to be unloaded is greater than or equal to a preset duration. Optionally, the preset duration can be manually preset or automatically adjusted by the electronic device according to the actual situation. For example, if there are 3 models, namely Model 1, Model 2, and Model 3, and if the unused durations of these 3 models are as shown in Table 4, and if the preset duration is 24 hours, since the unused duration of Model 1 is greater than the preset duration, and the unused durations of Model 2 and Model 3 are both less than the preset duration, then the model to be unloaded can be determined to be Model 10.

[0037] S503. Unload the model to be unloaded from the computing node. After determining the model to be unloaded, the electronic device can unload the model from the computing node. For example, if the model to be unloaded is model 1, the electronic device can unload model 10 from computing node 1. In this embodiment of the disclosure, the electronic device can determine the unused duration of each model in the loaded state in the computing node, and determine the model to be unloaded in the computing node according to the unused duration of each model, and then unload the model to be unloaded in the computing node. Through the above method, the electronic device can dynamically unload models that have not been called for a long time in any computing node, so as to release the resources of the computing node to the models that are called more frequently, thereby improving the resource utilization of the computing node. Through the model calling method described in Figures 2-5, multiple models can be run on a single processing instance, that is, multiple models can share a processing instance, which improves the resource utilization of the processing instance; and can realize dynamic loading and unloading of models. In typical scenarios involving thousands or more models, the model invocation method disclosed herein can deploy and invoke thousands of models, potentially saving up to 90% of the cost. Therefore, the technical solution disclosed herein can save costs and achieve high cost-effectiveness. Below, based on any of the above embodiments and in conjunction with Figure 6, the above model invocation method will be further described. Figure 6 is a schematic diagram of the structure of a model invocation system provided by an exemplary embodiment of this disclosure. Referring to Figure 6, the model invocation system includes an electronic device, a multi-model service, a model repository, and multiple public storage spaces. The electronic device may be equipped with a metadata-aware dynamic traffic routing mechanism; the electronic device may include a model distribution controller, a dynamic elastic scaler, a cache-aware scheduler, a Kubernetes scheduler, a model dynamic loading and unloading layer, and a model cache. The electronic device may be equipped with a metadata-aware dynamic traffic routing mechanism, which may include dynamic routing strategies. The electronic device can correctly route model invocation requests to containers with loaded target models through the metadata-aware dynamic traffic routing mechanism. The model distribution controller can dynamically load models into the container's memory or file path, and dynamically unload models from compute nodes based on model usage and compute node resource utilization to improve compute node resource utilization. For example, when the compute node's resource utilization exceeds a preset threshold, the model distribution controller can execute the model invocation method shown in Figure 5 to unload models that have not been invoked for a long time from the compute node, freeing up resources to load models that are invoked more frequently.The model distribution controller can invoke a dynamic elastic scaler to automatically increase the number of model replicas based on the load of each model, enabling it to handle more model call requests and cope with surges in request traffic. Alternatively, it can automatically reduce the number of model replicas when the load decreases, allowing for more efficient use of processing instance resources and reducing unnecessary resource consumption. Optionally, the model distribution controller can also invoke the dynamic elastic scaler to predict the model load over a future period using the historical load of each model, adjusting the number of model replicas in advance. Furthermore, it can invoke the dynamic elastic scaler to dynamically adjust resource allocation based on the resource usage of each compute node, ensuring the stable operation of each model on the compute node and providing stable performance. The dynamic elastic scaler maintains an appropriate number of replicas for each model, achieving high cost-effectiveness, ensuring the stability and high availability of model services, while minimizing compute node resources. The model distribution controller can also improve caching efficiency and reduce data loading latency through a cache-aware scheduler. Optionally, the model distribution controller can intelligently allocate target containers to compute nodes (i.e., target compute nodes) that already have relevant data caches by invoking the cache-aware scheduler. This reduces data transfer time and cost, accelerates the model invocation process through effective resource utilization and scheduling optimization, enhances the affinity between data and compute nodes, monitors the cache status of compute nodes in real time, and loads models onto compute nodes with existing relevant data caches. It can also assign appropriate priorities to each model based on its usage frequency and resource requirements, ensuring service for critical models. The cache-aware scheduler optimizes model invocation to provide faster data access speeds and reduce data access latency, thereby improving the efficiency of model invocation. The model distribution controller can invoke the Kubernetes scheduler to create target containers on target compute nodes. The model dynamic loading and unloading layer can include a model loading and unloading adapter and a model dynamic puller. The model loading and unloading adapter can be used to generate configuration files from model files and store the model files in a public storage space; the model dynamic puller can be used to retrieve model files from the model cache. A model cache may include a model cache controller, multiple model file resource objects, multiple cache engine resource objects, and multiple cache engines.For example, multiple model file resource objects can be designated as Model File Resource Object 1, Model File Resource Object 2, Model File Resource Object 3, ...; multiple cache engine resource objects can be designated as Cache Engine Resource Object 1, Cache Engine Resource Object 2, ...; multiple cache engines can be designated as Cache Engine 1, Cache Engine 2, ... The model cache controller can manage and monitor multiple model file resource objects and multiple cache engine resource objects. The model cache controller can be used to interact with the Kubernetes control plane and monitor the creation, updating, and deletion of model file resource objects and cache engine resource objects. The model cache controller ensures that the state of model file resource objects and cache engine resource objects is consistent with the expected state and takes corresponding actions as needed. Multiple cache engines can correspond to different distributed cache engine implementations. The model invocation system in this disclosure can provide an extension plugin that can support multiple data access engines. This is achieved by abstracting some common features, such as cache usage medium, quota, directory, etc., and the scalability features of different distributed cache engine implementation technologies. For example, a Filesystem in Userspace (FUSE) can be used to abstract, unify, and simplify distributed data access across multiple storage systems. In Kubernetes, the FUSE filesystem can be accessed by implementing a Container Storage Interface (CSI) plugin. A cache management mechanism can be included in the model cache. When a model is loaded, the cache management mechanism is triggered, automatically caching frequently accessed models to cache nodes closer to the compute nodes. Integrating a data acceleration engine can reduce model loading time and network overhead, especially when models are stored in remote data centers or cloud storage. A multi-model service can include multiple multi-model service runtimes. For example, a multi-model service can include multi-model service runtime 1, multi-model service runtime 2, and multi-model service runtime m (where m is an integer greater than or equal to 1). For any given multi-model service runtime, it can correspond to multiple compute nodes. Each compute node can include at least one processing instance, each processing instance can include at least one container, and each container can be used to run at least one model. Optionally, each processing instance can also include a model mesh agent.For example, a multi-model service runtime 1 may include 10 compute nodes, namely compute node 1, compute node 2, ... compute node 10. Compute node 1 may include processing instance 1 and processing instance 2. Processing instance 1 may include model mesh agent 1, container 1, and container 2. Processing instance 2 may include model mesh agent 2 and container 3. It should be noted that multiple multi-model service runtimes can share multiple compute nodes, that is, each compute node can support multiple multi-model service runtimes. For example, compute node 1 may be the compute node corresponding to multi-model service runtime 1, or it may be the compute node corresponding to multi-model service runtime 2. The model repository can be used to store model files of multiple models. The model repository can support data sources with various different protocols. For example, the model repository may store multiple model files such as model file 1, model file 2, and model file 3. Optionally, the model calling system may have multiple public storage spaces distributed among them. These multiple public storage spaces may be public storage space 1, public storage space 2, ... The public storage space can be used to store metadata and model files of models to be loaded. The process of requesting a model invocation is further explained in detail below, referring to Figure 6. This process may include steps ①②③④⑤⑥⑦⑧⑨⑩. In step ①, the electronic device can receive model invocation request 1. Model invocation request 1 can be used to request model 2 to recommend personalized products to user A. The electronic device can determine the status of model 2. If the status of model 2 is loaded, step ⑩ can be executed; if the status of model 2 is unloaded, steps ② to ⑩ can be executed. In step ②, the electronic device can determine whether model file 2 of model 2 exists in the model cache through the model distribution controller, the model loading and unloading adapter, and the model dynamic puller. Specifically, the electronic device can call the model loading and unloading adapter through the model distribution controller, and then call the model dynamic puller to determine whether model file 2 exists in the model cache. If model file 2 exists in the model cache, the electronic device can execute step ⑤, that is, retrieve model file 2 from the model cache using the model dynamic retrieval tool; if model file 2 does not exist in the model cache, the electronic device can execute steps ③ and ④. In step ③, the electronic device can determine the model file resource object 2 corresponding to model 2 through the model cache controller, and determine the storage information of model 2 in the model file resource object 2.Assuming the storage information of model 2 can be determined, including the target model's storage location in the model repository as storage location 1 and the storage type as type lo, in step ④, the electronic device can call cache engine 1 to obtain model file 2 through the model cache controller. Specifically, the electronic device can determine the cache engine as cache engine 1 based on the storage type being type 1, and can also determine the cache engine resource object 1 corresponding to cache engine 1. Then, based on the cache engine resource object 1, it can call the cache engine 1 to obtain model file 2 at storage location 1 in the model repository. Optionally, after obtaining model file 2, the electronic device can cache model file 2 in the model cache. Since model file 2 exists in the model cache or has just been cached, in step ⑤, the electronic device can obtain model file 2 from the model cache through the model dynamic puller. In step ⑥, the electronic device can generate configuration file 2 of model 2 based on model file 2 through the model load and unload adapter, and send configuration file 2 to the model distribution controller so that the model distribution controller can obtain configuration file 2. In step ⑦, the electronic device can determine at least one compute node corresponding to model 2 through the model distribution controller, and then determine the target container within that compute node. Specifically, the electronic device can determine 10 compute nodes corresponding to model 2 through the model distribution controller, namely compute node 1, compute node 2, ..., compute node 10. The electronic device can determine whether there is at least one available container among these 10 compute nodes through the model distribution controller. If there are 3 available containers among the 10 compute nodes, namely container 1, container 3, and container 21, and the container-related information of these 3 available containers is shown in Table 2, then the electronic device can determine container 3 as the target container based on the container-related information through the model distribution controller. If there are no available containers among the 10 compute nodes, the electronic device can call the cache-aware scheduler through the model distribution controller to determine the historical model running information of the 10 compute nodes in the metadata, and determine compute node 1 as the target compute node based on the historical model running information of the 10 compute nodes. The electronic device can call the Kubernetes scheduler through the model distribution controller to create the target container 22 in compute node 1.In step ⑧, the electronic device can store model file 2 in the public storage space lo closest to the computing node 1 where the target container is located through the model loading and unloading adapter. In step ⑨, the electronic device can load model 2 into the target container through the model distribution controller. Specifically, if the target container is container 22, the electronic device can obtain model file 2 from the public storage space 1 through the model distribution controller, and load model 2 into container 22 according to configuration file 2 and model file 2. In step ⑩, the electronic device can call model 2 for task processing. Specifically, if step ⑩ is executed after step ①, that is, if the state of model 2 is initially in the loading state, then it is possible that model 2 is running in multiple containers. Assuming that model 2 is running in both container 1 and container 3, the electronic device can send model call request 1 to container 1 according to the dynamic routing strategy in the metadata-based dynamic traffic routing mechanism, so that model 2 in container 1 can process model call request 1. If step 10 is executed after step 9, and if model 2 has just been loaded into the newly created container 22, then only container 22 is running model 2. The electronic device can then send model call request 1 to container 22 according to the dynamic routing strategy in the metadata-based dynamic traffic routing mechanism, so that model 2 in container 22 can process model call request 1. In this embodiment, the electronic device can obtain the model call request and determine the status of the target model. If the target model is in a loaded state, the electronic device can call the target model for task processing; if the target model is in an unloaded state, the electronic device can determine whether a model file exists in the model cache. If yes, the model file can be obtained from the model cache; otherwise, the model file can be obtained from the model repository. The electronic device can generate a configuration file based on the model file. The electronic device can determine at least one computing node corresponding to the target model, and determine the remaining capacity of each container in at least one computing node, as well as the target capacity required by the target model. Then, based on the remaining capacity and target capacity of each container in at least one computing node, the target container can be determined. Electronic devices can load the target model into the target container based on the configuration file and model file, and then call the target model to perform task processing.The model invocation system of this disclosure enables the hosting of multiple models within a single container, dynamically loading or unloading models from the container, and performing task processing on the target model, thereby enabling the invocation of a large number of models. Furthermore, the ability to dynamically load or unload models improves the resource utilization of containers, processing instances, and computing nodes. Figure 7 is a schematic diagram of a model invocation device provided by an exemplary embodiment of this disclosure. Please refer to Figure 7. The model invocation device 10 includes: an acquisition module 11, an invocation module 12, and a loading module 13. The acquisition module 11 is used to acquire a model invocation request, which requests the invocation of a target model for task processing. The invocation module 12 is used to, if the target model is in a loaded state, invoke the target model for task processing. The state of the target model is dynamically updated based on historical invocation information, and the loaded state indicates that the target model has been loaded into the container. The loading module 13 is used to, if the target model is in an unloaded state, load the target model. The unloaded state indicates that the target model has not been loaded into the container. The invocation module 12 is also used to invoke the target model for task processing. The model invocation device provided in this embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar and will not be repeated here. In one possible implementation, the loading module 13 is specifically configured to: obtain the configuration file and model file of the target model; determine a target container, the target container being used to load the target model; and load the target model in the target container according to the configuration file and the model file. In another possible implementation, the loading module 13 is specifically configured to: if the model file exists in a model cache, obtain the model file from the model cache; if the model file does not exist in the model cache, obtain the model file from a model repository, the model repository being used to store the model file; and generate the configuration file according to the model file. In another possible implementation, the loading module 13 is specifically configured to: determine the storage information of the target model, the storage information including the storage location and storage type of the target model in the model repository; determine a caching engine according to the storage type, the caching engine being used to obtain the model file corresponding to the storage type; and obtain the model file from the model repository through the caching engine.In one possible implementation, the loading module 13 is specifically configured to: determine at least one computing node corresponding to the target model, wherein the computing node includes at least one processing instance and at least one container corresponding to each processing instance; determine the remaining capacity of each container in the at least one computing node and the target capacity required by the target model; and determine the target container O based on the remaining capacity of each container in the at least one computing node and the target capacity. In another possible implementation, the loading module 13 is specifically configured to: determine whether there is at least one available container in the at least one computing node, wherein the remaining capacity of the available container is greater than or equal to the target capacity, based on the remaining capacity of each container in the at least one computing node and the target capacity; if yes, determine the target container among the at least one available container; if no, determine the target computing node among the at least one computing node and create the target container in the target computing node. In one possible implementation, the loading module 13 is specifically used to: acquire historical model running information for each computing node, the historical model running information including the identifier of the model run by the computing node within a historical time period and the model running time; and determine the target computing node among the at least one computing node based on the historical model running information of each computing node, the target computing node having run the target model within the historical time period, and the time interval between the model running time of the target model and the current time being the minimum. In one possible implementation, the loading module 13 is specifically used to: store the model file in a public storage space; and configure the model file in the public storage space according to the configuration file and the target container to load the target model in the target container. The model calling device provided in this embodiment can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, and will not be repeated here. Figure 8 is a schematic diagram of another model calling device provided in an exemplary embodiment of this disclosure. Please refer to Figure 8. Based on the embodiment shown in Figure 7, the model invocation device 10 further includes: a first determining module 14, a second determining module 15, and an unloading module 16. The first determining module 14 is used to determine the uninvoked duration of each model in the loaded state in the computing node. The second determining module 15 is used to determine the model to be unloaded in the computing node based on the uninvoked duration of each model, wherein the uninvoked duration of the model to be unloaded is greater than or equal to a preset duration. The unloading module 16 is used to unload the model to be unloaded in the computing node.The model invocation device provided in this disclosure can execute the technical solutions shown in the above method embodiments. Its implementation principle and beneficial effects are similar and will not be repeated here. Figure 9 is a schematic diagram of an electronic device provided in an exemplary embodiment of this disclosure. Referring to Figure 9, the electronic device 20 may include a processor 21 and a memory 22. Exemplarily, the processor 21 and the memory 22 are interconnected via a bus 23. The memory 22 stores computer execution instructions; the processor 21 executes the computer execution instructions stored in the memory 22, causing the processor 21 to execute the method as shown in the above method embodiments. Optionally, the electronic device shown in Figure 9 can be a physical server in a model invocation system. Accordingly, this disclosure provides a computer-readable storage medium storing computer execution instructions, which, when executed by a processor, are used to implement the method described in the above method embodiments. Accordingly, this disclosure may also provide a computer program product, including a computer program, which, when executed by a processor, can implement the method shown in the above method embodiments. Those skilled in the art should understand that the embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams. These computer program instructions may also be loaded onto a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams, whereby the instructions that execute on the computer or other programmable device provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams. In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory. Memory may include non-persistent memory in computer-readable media, random access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media. Computer-readable media include both permanent and non-persistent, removable and non-removable media; information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient media, such as modulated data signals and carrier waves. It should also be noted that the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.Unless otherwise specified, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, product, or apparatus that includes said element. The above description is merely an embodiment of this disclosure and is not intended to limit the scope of this disclosure. Various modifications and variations can be made to this disclosure by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of the claims of this disclosure.

Claims

CLAIM 1. A model invocation method, wherein, The method comprises the following steps: acquiring a model calling request, the model calling request being used for requesting to call a target model to process a task; if a state of the target model is a loaded state, calling the target model to process the task, the state of the target model being dynamically updated according to historical calling information of the target model, and the loaded state being used for indicating that the target model has been loaded into a container; if the state of the target model is an unloaded state, loading the target model and calling the target model to process the task, and the unloaded state being used for indicating that the target model has not been loaded into the container.

2. The method of claim 1, wherein, loading the target model comprises the following steps:

3. The method of claim 2, wherein, acquiring a configuration file and a model file of the target model; 4. The method according to claim 3, wherein, determining a target container, the target container being used for loading the target model; 5. The method according to any one of claims 2-4, wherein, according to the configuration file and the model file, loading the target model in the target container.

6. The method of claim 5, wherein, acquiring the configuration file and the model file of the target model comprises the following steps: 23 if the model file exists in a model cache, acquiring the model file in the model cache; if the model file does not exist in the model cache, acquiring the model file in a model storage, the model storage being used for storing model files; and generating the configuration file according to the model file. acquiring the model file in the model storage comprises the following steps: determining storage information of the target model, the storage information comprising a storage location and a storage type of the target model in the model storage; determining a cache engine according to the storage type, the cache engine being used for acquiring a model file corresponding to the storage type; and acquiring the model file in the model storage through the cache engine. determining the target container comprises the following steps: determining at least one computing node corresponding to the target model, the computing node comprising at least one processing instance and at least one container corresponding to each processing instance; determining a residual capacity of each container in the at least one computing node and a target capacity required by the target model; and determining the target container according to the residual capacity of each container in the at least one computing node and the target capacity. determining the target container according to the residual capacity of each container in the at least one computing node and the target capacity comprises the following steps: determining whether there is at least one available container in the at least one computing node according to the residual capacity of each container in the at least one computing node and the target capacity, the residual capacity of the available container being greater than or equal to the target capacity; and if yes, determining the target container in the at least one available container. if no, determining a target computing node in the at least one computing node and creating the target container in the target computing node.

7. The method of claim 6, wherein, The target computing node is determined in the at least one computing node, including: obtaining historical model running information of each computing node, the historical model running information including identification of a model run by the computing node in a historical period and a model running time; and determining the target computing node in the at least one computing node according to the historical model running information of each computing node, the target computing node having run the target model in the historical period and a time length between a model running time of the target model and a current time being the smallest.

8. The method of any one of claims 2-7, wherein, The target model is loaded in the target container according to the configuration file and the model file, including: storing the model file in a public storage space; and performing configuration processing on the model file in the public storage space according to the configuration file and the target container, so as to load the target model in the target container.

9. The method according to any one of claims 1-8, wherein, The method further includes: determining a time length during which each model in a loaded state in a computing node is not called; determining a model to be unloaded in the computing node according to the time length during which each model is not called, the time length during which the model to be unloaded is not called being greater than or equal to a preset time length; and unloading the model to be unloaded in the computing node.

10. A model invocation apparatus, wherein, The device includes an obtaining module, a calling module and a loading module, wherein the obtaining module is configured to obtain a model calling request, the model calling request being used to request calling a target model to perform task processing; the calling module is configured to call the target model to perform task processing if a state of the target model is a loaded state, the state of the target model being dynamically updated according to historical calling information of the target model, and the loaded state being used to indicate that the target model has been loaded into a container; and the loading module is configured to load the target model if a state of the target model is an unloaded state, the unloaded state being used to indicate that the target model has not been loaded into the container; and the calling module is further configured to call the target model to perform task processing.

11. An electronic device, wherein, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the electronic device to perform the method in any one of claims 1-9.

12. A computer-readable storage medium, wherein, The computer readable storage medium stores computer execution instructions, and when a processor executes the computer execution instructions, the method in any one of claims 1-9 is implemented. The computer program is executed by a processor to implement the method in any one of claims 1-9.

13. A computer program product comprising a computer program, wherein, ​ ​

Images