Artificial intelligence model management method, apparatus, device, and readable storage medium

Abstract

The application relates to an artificial intelligence model management method and device, equipment and a readable storage medium, and relates to the technical field of network intelligent operation and maintenance.The binary file, the configuration file and the metadata file corresponding to the target model are read.When it is determined through the metadata file that the target model has an added dependency relationship, it is determined whether a target process instance corresponding to the added dependency relationship exists in an independent instance group in a runtime module.If not, the added dependency relationship in the configuration file is added to a target folder where a Python installation package is located.The target folder is loaded to update the runtime environment, a new process instance is derived based on the updated runtime environment, and the new process instance is added to the independent instance group.The binary file and the configuration file are loaded on the independent instance group, the target model is loaded on the new process instance to run, and model deployment is completed, thereby improving the expansibility of the runtime environment and realizing unified deployment of different models.

Description

Technical Field

[0001] This application relates to the field of network intelligent operation and maintenance technology, and in particular to an artificial intelligence model management method, device, equipment and readable storage medium. Background Technology

[0002] Currently, the requirements for intelligent network operation and maintenance are becoming increasingly stringent, and the application of Artificial Intelligence (AI) is also gradually increasing. This not only further complicates intelligent network operation and maintenance, but also makes it difficult to guarantee the compatibility of uniformly deployed models because the runtime environments required for various AI application models differ, as do their required scalability. Therefore, improving the scalability of the runtime environment to achieve uniform deployment of models corresponding to different AI applications is a pressing issue that needs to be addressed. Summary of the Invention

[0003] This application provides an artificial intelligence model management method, apparatus, device, and readable storage medium to improve the scalability of the runtime environment, thereby enabling the unified deployment of models corresponding to different artificial intelligence applications.

[0004] Firstly, an artificial intelligence model management method is provided, including the following steps:

[0005] Read the binary file, configuration file, and metadata file corresponding to the target model;

[0006] When it is determined through the metadata file that the target model has a new dependency, it is determined whether there is a target process instance corresponding to the new dependency in the independent instance group within the runtime module.

[0007] If it does not exist, add the new dependency from the configuration file to the target folder where the Python installation package is located;

[0008] Load the target folder to update the runtime environment, derive a new process instance based on the updated runtime environment, and add the new process instance to the independent instance group;

[0009] Load the binary file and the configuration file on the independent instance group, so that the target model is loaded onto the new process instance and run to complete the model deployment.

[0010] In some embodiments, after the step of determining whether a target process instance corresponding to the newly added dependency exists in the independent instance group within the runtime module, the method further includes:

[0011] If present, the binary file and the configuration file are loaded on the independent instance group to load the target model onto the target process instance for execution.

[0012] In some embodiments, after the step of reading the binary file, configuration file, and metadata file corresponding to the target model, the method further includes:

[0013] When it is determined through the metadata file that there are no new dependencies in the target model, the binary file and the configuration file are loaded on the regular instance group within the runtime module, so that the target model is loaded and run on the process instance corresponding to the target model in the regular instance group.

[0014] In some embodiments, the binary file and the configuration file are loaded by dynamically importing importlib.

[0015] In some embodiments, after the step of loading the binary file and the configuration file on a separate instance group, causing the target model to be loaded and run on a new process instance, the method further includes:

[0016] When a training request is received, the training method in the configuration file and the target model's data in the actual deployment environment are used to train the target model online.

[0017] The target model is updated based on the training accuracy.

[0018] In some embodiments, the method further includes:

[0019] When an inference request is received, the first process instance is determined based on the model label in the inference request, and the loaded model corresponding to the first process instance is invoked for inference.

[0020] In some embodiments, the method further includes:

[0021] Obtain the real-time available resources of physical nodes;

[0022] The target physical node is determined based on the relationship between real-time available resources and resource thresholds, and is used to execute model deployment requests, model training requests, or model inference requests.

[0023] Secondly, an artificial intelligence model management device is provided, comprising:

[0024] The reading module is used to read the binary file, configuration file, and metadata file corresponding to the target model.

[0025] The management module is used to determine whether a target process instance corresponding to the new dependency exists in the independent instance group within the runtime module when a new dependency is determined through the metadata file; if not, the new dependency in the configuration file is added to the target folder where the Python installation package is located; the target folder is loaded to update the runtime environment, a new process instance is derived based on the updated runtime environment, and the new process instance is added to the independent instance group.

[0026] The runtime module is used to load the binary file and the configuration file on a separate instance group, so that the target model is loaded onto a new process instance and run to complete the model deployment.

[0027] Thirdly, an artificial intelligence model management device is provided, comprising: a memory and a processor, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the aforementioned artificial intelligence model management method.

[0028] Fourthly, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions, which, when executed by a computer, cause the computer to perform the aforementioned artificial intelligence model management method.

[0029] This application provides an artificial intelligence model management method, apparatus, device, and readable storage medium, including reading a binary file, configuration file, and metadata file corresponding to a target model; when it is determined through the metadata file that the target model has a new dependency, determining whether there is a target process instance corresponding to the new dependency in the independent instance group within the runtime module; if not, adding the new dependency in the configuration file to the target folder where the Python installation package is located; loading the target folder to update the runtime environment, deriving a new process instance based on the updated runtime environment, and adding the new process instance to the independent instance group; loading the binary file and configuration file on the independent instance group, so that the target model is loaded onto the new process instance and runs, thereby completing the model deployment. This application determines whether to add an extension dependency to a specified runtime by identifying whether the model's metadata carries a new dependency that represents the need for runtime environment extension, and deploys and runs the model with the new dependency on the independent instance group, thereby improving the scalability of the runtime environment and achieving unified deployment of different models; in addition, it also starts the corresponding process by specifying the Python path to isolate different runtime environments, thereby effectively avoiding the impact on the original process. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 A flowchart illustrating an artificial intelligence model management method provided in an embodiment of this application;

[0032] Figure 2 This is a schematic diagram of the AI ​​engine structure provided in an embodiment of this application;

[0033] Figure 3 This is a schematic diagram of the inference training service architecture provided in the embodiments of this application;

[0034] Figure 4 This is a schematic diagram of the structure of an artificial intelligence model management device provided in an embodiment of this application. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0036] This application provides an artificial intelligence model management method, apparatus, device, and readable storage medium, which can improve the scalability of the runtime environment and thus realize the unified deployment of models corresponding to different artificial intelligence applications.

[0037] Figure 1 This application provides an artificial intelligence model management method, which includes the following steps:

[0038] Step S10: Read the binary file, configuration file, and metadata file corresponding to the target model;

[0039] As an example, in this embodiment, an AI engine will be used to integrate the model management service and the model inference and training service. The model management service in this embodiment is mainly used to read and parse model package files, deploy models to specified process instances based on model tags, decide whether to add new independent instances based on new model dependencies, and schedule inference and training requests based on the resource availability of physical nodes. The model inference and training service is used to support the model's runtime services. It exposes corresponding interfaces to the model management service by loading the model initialization method, training method, and inference method from the configuration file. When online training is required, it executes training by calling the training interface and obtains the training accuracy, which is then fed back to the model management service to decide whether to update the model.

[0040] The control system for AI applications such as optical performance degradation applications, traffic prediction applications, and hidden danger analysis applications is independent of the AI ​​engine provided in this embodiment. AI applications can interact with the AI ​​engine via HTTP (Hypertext Transfer Protocol) or FTP (File Transfer Protocol) for inference and online training. See also... Figure 2 As shown, the AI ​​engine includes a service implementation module and a runtime module. The service implementation module includes a model deployment unit, a model inference unit, a model training unit, and a model management unit, etc. That is, the model management service and the model inference and training service are implemented through the model deployment unit, the model inference unit, the model training unit, and the model management unit (which is used for deployment triggering operations and AI engine interface). The runtime module carries the basic environment of the model inference and training service, has complete runtime support, and can add model dependencies in a hot deployment and incremental manner according to the new dependency requirements.

[0041] It should be noted that, due to potential conflicts and upgrade incompatibility between newly added dependencies and existing dependencies, directly modifying existing dependencies based on new ones could render existing models unusable. Therefore, to address this issue, the Runtime module in this embodiment includes not only a runtime environment for running regular models (i.e., models with basic dependencies) but also a runtime environment for running new models (i.e., models with new dependencies). Specifically, the Runtime module in this embodiment includes not only a regular instance group for running basic models but also a newly added independent instance group for running new models. The regular instance group includes the base toolkit and deep learning training frameworks (such as Tenorflow and PyTorch). It is understood that when the AI ​​engine starts, it will first launch the regular instance group to run the corresponding model according to the pre-set number of instances, and then launch either the regular instance group or the independent instance group according to the model deployment request.

[0042] In this embodiment, a model package file is constructed for each AI application's corresponding model. This model package file includes a binary file corresponding to the model, a configuration file, and a metadata file. The metadata file primarily stores information describing the model, such as whether it carries new dependencies, the model version, model labels, and model accuracy (i.e., model precision). The configuration file primarily stores the implementation methods of the model, such as the `init` method, the `train` method, and the `predict` method, as well as the model calling files. For models with new dependencies, the configuration file also stores the paths to those new dependencies.

[0043] Understandably, metadata files can use 0 and 1 to indicate whether new dependencies are carried. If the target model's dependency package is different from existing third-party dependency packages, it indicates that the target model has new dependencies, and the value of the item in the metadata file regarding whether new dependencies are carried is set to 1; if the target model's dependency package is the same as existing third-party dependency packages, it indicates that the target model does not have new dependencies, and the value of the item in the metadata file regarding whether new dependencies are carried is set to 0.

[0044] Step S20: When it is determined through the metadata file that there is a new dependency relationship in the target model, determine whether there is a target process instance corresponding to the new dependency relationship in the independent instance group within the runtime module;

[0045] In this embodiment, after the step of determining whether there is a target process instance corresponding to the newly added dependency in the independent instance group within the runtime module, the method further includes:

[0046] If present, the binary file and the configuration file are loaded on the independent instance group to load the target model onto the target process instance for execution.

[0047] As an example, it should be understood that this embodiment can determine whether the model is a regular model or a new model with new dependencies by the value of the item regarding whether new dependencies are carried in the metadata file; for example, if the value of the item regarding whether new dependencies are carried in the metadata file is 1, it indicates that the target model has new dependencies, indicating that the target model is a new model; while if the value is 0, it indicates that the target model does not have new dependencies, indicating that the target model is a regular model.

[0048] It should be noted that for deployed models with new dependencies, the new process instances derived from the new dependencies will be added to the independent instance group. This will allow subsequent models with the same new dependencies to be directly assigned to the corresponding runnable instances during deployment, thereby improving deployment efficiency.

[0049] Therefore, when it is determined that the target model has new dependencies, the new dependencies will be obtained through the paths of the new dependencies stored in the configuration file. It will then be checked whether there is a runnable target process instance corresponding to the new dependency in the independent instance group within the Runtime module, that is, whether there is an instance in the independent instance group that can host the target model. If it exists, it means that a model with the same new dependencies as the target model has been deployed before. In this case, the target model will be directly assigned to the target process instance, that is, the binary file and configuration file of the target model will be directly loaded on the independent instance group so that the target model can be loaded and run on the target process instance, i.e., run as part of the process. If it does not exist, the Runtime needs to be extended, that is, new environment paths and dependencies will be added independently on the original basis, and the new model inference training service will be started on it to deploy the new model.

[0050] Furthermore, after the step of reading the binary file, configuration file, and metadata file corresponding to the target model, the method further includes:

[0051] When it is determined through the metadata file that there are no new dependencies in the target model, the binary file and the configuration file are loaded on the regular instance group within the runtime module, so that the target model is loaded and run on the process instance corresponding to the target model in the regular instance group.

[0052] As an example, in this embodiment, when it is determined through the metadata file that the target model has no new dependencies, it indicates that the target model is a regular model. Then, the binary file and configuration file of the target model can be loaded on the regular instance group in the Runtime module through the model tag, so that the target model can be loaded onto the process instance corresponding to the target model in the regular instance group for execution.

[0053] Step S30: If it does not exist, add the new dependency in the configuration file to the target folder where the Python installation package is located;

[0054] As an example, in this embodiment, when there is no runnable target process instance in the independent instance group that corresponds to the new dependency, it means that no model with the same new dependency as the target model has been deployed before. In this case, the Runtime needs to be extended, that is, new environment paths and dependencies are added independently on the original basis. In other words, the new dependency in the target model's configuration file is added to the target folder where the Python installation package is located, so that the current target model can use different Runtime environments.

[0055] Step S40: Load the target folder to update the runtime environment, derive a new process instance based on the updated runtime environment, and add the new process instance to the independent instance group;

[0056] As an example, it's important to note that the traditional method of starting a process involves setting Python environment variables and then launching the process based on those variables. However, changes to the runtime environment can affect all hosted processes. This embodiment, however, does not set environment variables. Instead, it starts the process by specifying the Python path. The model management service uses the new runtime environment configuration to derive new process instances, thus isolating multiple different runtime environments and effectively avoiding impact on existing processes. Specifically, it loads the target folder containing the new dependencies to update the runtime environment, derives new process instances based on the updated runtime environment, and adds the new process instances to an independent instance group.

[0057] Step S50: Load the binary file and the configuration file on the independent instance group, so that the target model is loaded onto the new process instance to run, thereby completing the model deployment.

[0058] As an example, in this embodiment, after deriving a new process instance using a new runtime environment configuration, model deployment is achieved through the model management service. Specifically, the binary file and configuration file of the target model are loaded onto an independent instance group, allowing the target model to run on the new process instance, thus completing model deployment and starting the AI ​​engine's model inference training service. Therefore, this embodiment improves the scalability of the runtime environment by deriving new process instances for dependencies on the new runtime environment, ensuring that newly added dependencies do not affect the original runtime environment and achieving unified deployment of different models.

[0059] Furthermore, the binary file and the configuration file are loaded by dynamically importing importlib.

[0060] As an example, in this embodiment, when loading the model's binary file and configuration file to load the target model onto the process instance and run it as part of the process, it is preferable to load the binary file and configuration file by dynamically importing importlib, so that the service does not need to be restarted when the model is deployed or updated.

[0061] Furthermore, after the step of loading the binary file and the configuration file on the independent instance group, so that the target model is loaded and run on the new process instance, the method further includes:

[0062] When a training request is received, the training method in the configuration file and the target model's data in the actual deployment environment are used to train the target model online.

[0063] The target model is updated based on the training accuracy.

[0064] Exemplary, see Figure 3 As shown, this embodiment will use Flask (a web micro-framework written in Python) to publish the model training and inference service as an HTTP service, exposing initialization interfaces, inference interfaces, and training interfaces to the model management service; the deployed target model will be imported as a module into the model inference and training service through importlib, and the init method corresponding to the tags (such as m1 tag and m2 tag) in the metadata file will be loaded to initialize the target model, and the corresponding train method and predict method will be loaded to provide training and inference services.

[0065] It is understandable that current models are often obtained through offline training. However, the actual application scenarios of these models and the offline scenarios usually differ, resulting in insufficient accuracy of the models in real-world applications—that is, the offline-trained models are not well-suited to the actual scenarios. The AI ​​engine in this embodiment, however, can improve the accuracy of the model through online training. Specifically, the online training in this embodiment reuses the code from the model training process, i.e., the code is moved to a configuration file, requiring only modification of the read data. For example, when a training request is received, the model management service, based on the training configuration requirements (such as training by precision or training by time), calls the `train` method in the loaded configuration file corresponding to the target model and the corresponding new data in the actual deployment environment for online training. Then, it compares the accuracy obtained from online training with the accuracy of the current model stored in the metadata file. When the accuracy obtained from online training is higher, the online-trained model replaces the current model while retaining the historical version. Thus, this embodiment utilizes new data to correct situations where the original model is not adapted to new scenarios.

[0066] Furthermore, the method also includes:

[0067] When an inference request is received, the first process instance is determined based on the model label in the inference request, and the loaded model corresponding to the first process instance is invoked for inference.

[0068] As an example, in this embodiment, when the model management service receives an inference request, it will analyze the model tags in the inference request to determine the corresponding process instance based on the model tags, and assign it to the process instance, while calling the corresponding loaded model for inference.

[0069] Furthermore, the method also includes:

[0070] Obtain the real-time available resources of physical nodes;

[0071] The target physical node is determined based on the relationship between real-time available resources and resource thresholds, and is used to execute model deployment requests, model training requests, or model inference requests.

[0072] As an example, it should be understood that the system resources consumed by a model vary depending on its running state. For instance, high resource requirements during model inference or training may affect the use of other processes, thus often leading to resource contention during model runtime. In this embodiment, the model management service stores resource information for each physical node (each physical node may have multiple instance processes). When there is a new model deployment, model training request, or model inference request, the model management service is invoked to obtain the latest resource status of each physical node. Based on a strategy that considers the relationship between real-time available resources and resource thresholds, a suitable physical node is selected to operate the new model deployment, model training, or model inference.

[0073] Understandably, since the resource requirements for model deployment, model training, and model inference are different, the corresponding resource thresholds for model deployment, model training, and model inference are also different. For example, the resource threshold for model deployment is A, the resource threshold for model training is B, and the resource threshold for model inference is C. Taking model training as an example, assuming physical nodes include X and Y, if the real-time available resources of both X and Y are greater than B, then either X or Y can be chosen for model training; if the real-time available resources of X are greater than B and Y are less than B, then model training is performed through X; if the real-time available resources of both X and Y are less than B, i.e., when node resources are insufficient, then the model management service needs to remove the model with the fewest recent calls to release new resources for model training. If the model with the fewest recent calls is in X, then after that model is removed, model training is performed through X. Therefore, this embodiment avoids resource contention by managing inference and training resources, thereby ensuring the efficiency of inference and training.

[0074] It should be noted that the step numbers in the embodiments of this application do not limit the order of operations in the technical solution of this application.

[0075] This application also provides an artificial intelligence model management device, including:

[0076] The reading module is used to read the binary file, configuration file, and metadata file corresponding to the target model.

[0077] The management module is used to determine whether a target process instance corresponding to the new dependency exists in the independent instance group within the runtime module when a new dependency is determined through the metadata file; if not, the new dependency in the configuration file is added to the target folder where the Python installation package is located; the target folder is loaded to update the runtime environment, a new process instance is derived based on the updated runtime environment, and the new process instance is added to the independent instance group.

[0078] The runtime module is used to load the binary file and the configuration file on a separate instance group, so that the target model is loaded onto a new process instance and run to complete the model deployment.

[0079] Furthermore, the management module is also used for:

[0080] If present, the binary file and the configuration file are loaded on the independent instance group to load the target model onto the target process instance for execution.

[0081] Furthermore, the management module is also used for:

[0082] When it is determined through the metadata file that there are no new dependencies in the target model, the binary file and the configuration file are loaded on the regular instance group within the runtime module, so that the target model is loaded and run on the process instance corresponding to the target model in the regular instance group.

[0083] Furthermore, the binary file and the configuration file are loaded by dynamically importing importlib.

[0084] Furthermore, the device also includes a training module, which is used for:

[0085] When a training request is received, the training method in the configuration file and the target model's data in the actual deployment environment are used to train the target model online.

[0086] The target model is updated based on the training accuracy.

[0087] Furthermore, the device also includes a reasoning module, which is used for:

[0088] When an inference request is received, the first process instance is determined based on the model label in the inference request, and the loaded model corresponding to the first process instance is invoked for inference.

[0089] Furthermore, the device also includes a processing module for:

[0090] Obtain the real-time available resources of physical nodes;

[0091] The target physical node is determined based on the relationship between real-time available resources and resource thresholds, and is used to execute model deployment requests, model training requests, or model inference requests.

[0092] It should be noted that those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the above-described apparatus and modules can be referred to the corresponding processes in the aforementioned embodiments of the artificial intelligence model management method, and will not be repeated here.

[0093] The apparatus provided in the above embodiments can be implemented as a computer program, which can be used in, for example... Figure 4 The artificial intelligence model shown runs on the management device.

[0094] This application also provides an artificial intelligence model management device, including: a memory, a processor, and a network interface connected via a system bus. The memory stores at least one instruction, which is loaded and executed by the processor to implement all or part of the steps of the aforementioned artificial intelligence model management method.

[0095] The network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0096] A processor can be a CPU, or other general-purpose processors, DSPs (Digital Signal Processors), ASICs (Application Specific Integrated Circuits), FPGAs (Field Programmable Gate Arrays), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor, or any conventional processor. The processor is the control center of a computer device, connecting all parts of the computer device through various interfaces and lines.

[0097] Memory can be used to store computer programs and / or modules. The processor performs various functions of the computer device by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory. Memory can primarily include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function, etc.; the data storage area may store data created based on system usage, etc. Furthermore, memory may include high-speed random access memory (RAM), and may also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, SMC (Smart Media Card), SD (Secure Digital) cards, flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0098] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements all or part of the steps of the aforementioned artificial intelligence model management method.

[0099] The embodiments of this application can implement all or part of the aforementioned processes, or they can be accomplished by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various methods described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, ROM (Read-Only memory), RAM (Random Access memory), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0100] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, servers, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0101] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0102] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0103] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for managing artificial intelligence models, characterized in that, Includes the following steps: Read the binary file, configuration file, and metadata file corresponding to the target model; When it is determined through the metadata file that the target model has a new dependency, it is determined whether there is a target process instance corresponding to the new dependency in the independent instance group within the runtime module. If they exist, the binary file and the configuration file are loaded on the independent instance group so that the target model is loaded and run on the target process instance; If it does not exist, add the new dependency from the configuration file to the target folder where the Python installation package is located; Load the target folder to update the runtime environment, derive a new process instance based on the updated runtime environment, and add the new process instance to the independent instance group; Load the binary file and the configuration file on the independent instance group, so that the target model is loaded onto the new process instance and run to complete the model deployment.

2. The artificial intelligence model management method as described in claim 1, characterized in that, After the step of reading the binary file, configuration file, and metadata file corresponding to the target model, the method further includes: When it is determined through the metadata file that there are no new dependencies in the target model, the binary file and the configuration file are loaded on the regular instance group within the runtime module, so that the target model is loaded and run on the process instance corresponding to the target model in the regular instance group.

3. The artificial intelligence model management method as described in claim 1 or 2, characterized in that: The binary file and the configuration file are loaded by dynamically importing importlib.

4. The artificial intelligence model management method as described in claim 1, characterized in that, Following the step of loading the binary file and the configuration file on a separate instance group, thereby loading the target model to run on a new process instance, the method further includes: When a training request is received, the training method in the configuration file and the target model's data in the actual deployment environment are used to train the target model online. The target model is updated based on the training accuracy.

5. The artificial intelligence model management method as described in claim 4, characterized in that, The method further includes: When an inference request is received, the first process instance is determined based on the model label in the inference request, and the loaded model corresponding to the first process instance is invoked for inference.

6. The artificial intelligence model management method as described in claim 5, characterized in that, The method further includes: Obtain the real-time available resources of physical nodes; The target physical node is determined based on the relationship between real-time available resources and resource thresholds, and is used to execute model deployment requests, model training requests, or model inference requests.

7. An artificial intelligence model management device, characterized in that, include: The reading module is used to read the binary file, configuration file, and metadata file corresponding to the target model. The management module is used to determine whether a target process instance corresponding to the new dependency exists in the independent instance group within the runtime module when a new dependency is determined through the metadata file. If it exists, the binary file and the configuration file are loaded onto the independent instance group so that the target model is loaded and run on the target process instance. If it does not exist, the new dependency in the configuration file is added to the target folder where the Python installation package is located. The target folder is loaded to update the runtime environment, a new process instance is derived based on the updated runtime environment, and the new process instance is added to the independent instance group. The runtime module is used to load the binary file and the configuration file on a separate instance group, so that the target model is loaded onto a new process instance and run to complete the model deployment.

8. An artificial intelligence model management device, characterized in that, include: A memory and a processor, wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the artificial intelligence model management method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions that, when executed by a computer, cause the computer to perform the artificial intelligence model management method according to any one of claims 1 to 6.

Images