Artificial intelligence model management method, apparatus, and system
By adopting a phased storage and verification approach to AI model management, the problem of misdeployment without sufficient verification in AI model management is solved, ensuring the stability and security of deployed models and reducing business risks.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING QIYI CENTURY SCI & TECH CO LTD
- Filing Date
- 2026-02-25
- Publication Date
- 2026-06-05
AI Technical Summary
Existing AI model management methods are prone to misdeployment without sufficient validation, leading to business risks and operational losses.
By storing AI models and their metadata in different databases in stages for training, testing, delivery, and deployment, we ensure that fully validated models are deployed only in production environments.
This effectively avoids the deployment of AI models that have not been fully validated, improves the stability and security of model management, and reduces business risks.
Smart Images
Figure CN122153367A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of AI model management, and more particularly to an AI model management method, apparatus, and system. Background Technology
[0002] For AI models (Artificial Intelligence Models) in fields such as autonomous driving, medical diagnosis, and financial risk control, frequent iterations and updates of AI model versions are required, and the stability and security requirements of AI models are higher. Currently, the management of AI models in these fields generally adopts a single storage pool management model. AI models involved in the development environment (covering the training and verification process of AI models) and AI model assets involved in the production environment (covering the deployment process of the corresponding services of AI models) are mixed together and stored in the same database. This is prone to cross-contamination between the development and production environments. For example, in the production environment (where the services corresponding to AI models are deployed), an AI model that has not been fully verified may be mistakenly obtained from the database and deployed. This may lead to online service failures due to the AI model's performance, security, or compliance not meeting the requirements, resulting in business risks and operational losses.
[0003] In summary, existing AI model management methods are prone to misdeployment of AI models that have not yet been fully validated. Summary of the Invention
[0004] This application provides an AI model management method, an AI model management device, and an AI model management system to address the problem that existing AI model management methods are prone to misdeployment of AI models that have not yet been fully validated.
[0005] In a first aspect, this application provides an AI model management method, the method comprising: binding an AI model and its metadata to a first database, wherein the AI model is a base model and the metadata includes at least a name and a version number; responding to the metadata in a training task instruction, selecting a corresponding AI model from the first database, training the AI model to obtain a trained AI model, testing and verifying the trained AI model, and when the test and verification pass, binding the metadata of the trained AI model and the trained AI model to a second database; responding to the metadata in a delivery task instruction, selecting a corresponding AI model from the second database, performing delivery verification on the AI model, obtaining a service corresponding to the AI model when the delivery verification passes, and binding the metadata of the AI model and the service to a third database; and responding to the metadata in a deployment instruction, selecting a corresponding service from the third database and deploying the service to a target device.
[0006] Optionally, when the test verification passes, after binding and storing the metadata of the trained AI model and the trained AI model in the second database, the method further includes: binding and storing the metadata of the trained AI model and the trained AI model in the first database.
[0007] Optionally, upon successful test verification, before binding and storing the metadata of the trained AI model and the trained AI model in the second database, the method further includes: obtaining the name and version number of the AI model from the metadata in the training task instruction; obtaining the version chain of the AI model from the fourth database based on the name of the AI model, the version chain being used to record the lineage relationship and version numbers of AI models derived from the base model of the AI model; determining the version number of the trained AI model based on the version number of the AI model, the version chain of the AI model, and a preset version number encoding rule; replacing the version number of the AI model with the version number of the trained AI model in the metadata of the training task instruction to obtain the metadata of the trained AI model, and recording the version number of the trained AI model in the version chain of the AI model; binding the version chain of the AI model with the name of the AI model and storing it in the fourth database.
[0008] Optionally, after the AI model is delivered and verified, the method further includes: obtaining the name of the AI model from the metadata in the delivery task instruction; obtaining the version chain of the AI model from the fourth database according to the name of the AI model; setting a delivery verification passed flag for the version number of the AI model in the version chain when the delivery verification is passed, and setting a delivery verification failed flag for the version number of the AI model in the version chain when the delivery verification is failed.
[0009] Optionally, after deploying the service to the target device, the method includes: periodically detecting whether the service is in an abnormal operating state; if the service is in the abnormal operating state, obtaining the name of the AI model corresponding to the service and the version number of the AI model corresponding to the service from the metadata in the deployment instruction; obtaining the version chain of the AI model corresponding to the service from a fourth database based on the name of the AI model corresponding to the service; determining a rollback version number based on the version number of the AI model and the version chain of the AI model, wherein the rollback version number is the first version number of the AI model in the version chain of the AI model that has the delivery verification passed identifier along the version backtracking direction; obtaining the corresponding service from the third database based on the name of the AI model and the rollback version number; and deploying the service to the target device.
[0010] Optionally, the trained AI model is tested and verified, including: testing and verifying the trained AI model to obtain multiple test verification indicators, including at least: test accuracy indicator, inference latency indicator, memory usage indicator, and robustness indicator; and determining that the test verification is passed if at least one of the test verification indicators meets the corresponding test verification indicator conditions.
[0011] Optionally, the AI model is delivered and verified. Upon successful delivery verification, the corresponding service is obtained. This includes: testing the AI model in a shadow environment using simulated production traffic and obtaining multiple business verification metrics, including at least throughput and stability metrics; obtaining the manual verification result of the AI model, which is based at least on business alignment, data deviation, and compliance review; performing a delivery operation on the AI model, which involves encapsulating the AI model into a service and performing interface verification on the service; and determining that the delivery verification is successful and the service is obtained when all multiple business verification metrics meet their corresponding business verification metric conditions, the manual verification result is passed, and the interface verification is passed.
[0012] Optionally, after binding the AI model and its metadata to the first database, the method includes: backing up and storing the AI model and its metadata to an object storage service system.
[0013] Secondly, this application provides an AI model management device, comprising: a model management unit, configured to bind and store an AI model and its metadata in a first database, wherein the AI model is a basic model and the metadata includes at least: name and version number; a training unit, configured to, in response to the metadata in a training task instruction, select a corresponding AI model from the first database, train the AI model to obtain a trained AI model, test and verify the trained AI model, and, upon successful test and verification, bind and store the metadata of the trained AI model and the trained AI model in a second database; an algorithm delivery unit, configured to, in response to the metadata in a delivery task instruction, select a corresponding AI model from the second database, perform delivery verification on the AI model, obtain a service corresponding to the AI model upon successful delivery verification, and bind and store the metadata of the AI model and the service in a third database; and a service registration unit, configured to, in response to the metadata in a deployment instruction, select a corresponding service from the third database and deploy the service to a target device.
[0014] Thirdly, this application provides an AI model management system, which includes: a first database, a second database, a third database, and a fourth database, and a management module. The management module is used to implement the steps of any of the AI model management methods described above.
[0015] In this embodiment, in the development environment, the trained AI model is tested and verified. Upon successful testing and verification, the metadata of the trained AI model and the trained AI model itself are bound and stored in a second database. Responding to the metadata in the delivery task instruction, the corresponding AI model is selected from the second database, and the AI model is delivered and verified. Upon successful delivery verification, the service corresponding to the AI model is obtained, and the metadata of the AI model and the service are bound and stored in a third database. In the production environment, responding to the metadata in the deployment instruction, the corresponding service is selected from the third database, and the service is deployed to the target device. That is, the second database only stores AI models that have passed testing and verification, and the third database only stores services corresponding to AI models that have passed both testing and delivery verification. Since the service is selected from the third database in response to the metadata in the deployment instruction in the production environment, the service deployed to the target device must be a service corresponding to an AI model that has passed both testing and delivery verification; that is, the service deployed to the target device must be a service corresponding to a fully verified AI model. This solves the problem in existing AI model management methods where AI models are easily misdeployed before they have passed sufficient verification. Attached Figure Description
[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0018] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.
[0019] Figure 1 A flowchart illustrating an AI model management method provided in an embodiment of this application; Figure 2 A schematic diagram of a version chain provided for an embodiment of this application; Figure 3 A schematic diagram of another version chain provided for an embodiment of this application; Figure 4 A structural block diagram of an AI model management device provided in an embodiment of this application; Figure 5 A structural block diagram of an AI model management system provided in an embodiment of this application; The symbols in the attached image are explained as follows: 10. First database; 20. Second database; 30. Third database; 40. Fourth database; 50. Object storage service system; 60. Management module. Detailed Implementation
[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0021] The following disclosure provides numerous different embodiments or examples for implementing various structures of the invention. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of the invention. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.
[0022] The following definitions are provided for the terms used: Base model: refers to a pre-trained model built using classic model architectures such as ResNet (Residual Network) and BERT (Bidirectional Encoder Representations from Transformers). Its core includes the initial parameters of pre-training and the complete network structure design, serving as a benchmark for subsequent fine-tuning or customized development for specific tasks.
[0023] Reasoning task: refers to the process of using deployed AI models to predict, classify, or make decisions about new data.
[0024] To address the technical problem that existing AI model management methods are prone to misdeployment of AI models before they have been fully validated, this application provides an AI model management method, an AI model management device, and an AI model management system, which can solve the problem that existing AI model management methods are prone to misdeployment of AI models before they have been fully validated.
[0025] Figure 1 An AI model management method provided in this application embodiment, such as Figure 1 As shown, the above method includes: Step S101: Bind the AI model and its metadata to the first database. The AI model is the base model, and the metadata includes at least: name and version number. Step S102: In response to the metadata in the training task instruction, select the corresponding AI model from the first database, train the AI model to obtain the trained AI model, test and verify the trained AI model, and when the test and verification are passed, bind the metadata of the trained AI model and the trained AI model and store them in the second database. It should be noted that training AI models is an existing technology, therefore, this application will not provide a detailed explanation.
[0026] Step S103: In response to the metadata in the delivery task instruction, select the corresponding AI model from the second database, perform delivery verification on the AI model, obtain the service corresponding to the AI model when the delivery verification is passed, and store the metadata of the AI model and the service bound to the third database. In step S104, in response to the metadata in the deployment instruction, the corresponding service is selected from the third database and deployed to the target device.
[0027] For example, since the services corresponding to the deployed AI models do not require a very large amount of computation when performing inference tasks, the target device is usually a CPU (Central Processing Unit) node to reduce costs.
[0028] Through the above embodiments, in the development environment, the trained AI model is tested and verified. When the test and verification pass, the metadata of the trained AI model and the trained AI model itself are bound and stored in a second database. In response to the metadata in the delivery task instruction, the corresponding AI model is selected from the second database, and the AI model is delivered and verified. When the delivery verification passes, the service corresponding to the AI model is obtained, and the metadata of the AI model and the service are bound and stored in a third database. In the production environment, in response to the metadata in the deployment instruction, the corresponding service is selected from the third database, and the service is deployed to the target device. That is, the second database only stores AI models that have passed the test and verification, and the third database only stores services corresponding to AI models that have passed both the test and verification and the delivery verification. Since the corresponding service is selected from the third database in response to the metadata in the deployment instruction in the production environment, the service deployed to the target device must be the service corresponding to an AI model that has passed both the test and verification and the delivery verification. In other words, the service deployed to the target device must be the service corresponding to a fully verified AI model. This solves the problem that existing AI model management methods are prone to misdeploying AI models that have not yet passed sufficient verification.
[0029] Upon successful verification of the above tests, after binding and storing the metadata of the trained AI model and the trained AI model in the second database, the method further includes: The metadata of the trained AI model and the trained AI model itself are bound together and stored in the first database.
[0030] For example, considering the need for frequent iterations and updates of the AI model version, in this case, it is necessary to iteratively train the AI model. Therefore, the metadata of the trained AI model and the trained AI model are bound and stored in the first database. When the version of the AI model needs to be updated, the trained AI model is selected from the first database and the trained AI model is trained again.
[0031] For example, the metadata of an AI model also includes: model description information, model permission information, and performance indicator information. The model description information includes: the function, purpose, and construction method of the AI model. The model permission information includes: read permission (e.g., the parameters of the AI model can be read) and write permission (e.g., the parameters of the AI model cannot be written). The performance indicator information includes: test accuracy indicators, memory usage indicators, inference latency indicators, etc.
[0032] Upon successful verification of the above tests, before binding and storing the metadata of the trained AI model and the trained AI model in the second database, the above method further includes: Obtain the name and version number of the AI model from the metadata in the training task instructions. Based on the name of the AI model, the version chain of the AI model is obtained from the fourth database. The version chain is used to record the genealogical relationship and version number of the AI models derived from the base model of the AI model. It should be noted that AI models derived from the same base model have the same name, and the version chains of AI models derived from the same base model are the same.
[0033] For example, Figure 2 An example version chain is given. Figure 2 The arrows in the text indicate the direction of derivation, such as... Figure 2 As shown, the version chain starts with the version number of the basic model (V1.0.0), and the version numbers of the first-generation AI models derived from the basic model (V2.0.0, V3.0.0, V4.0.0), the version numbers of the second-generation AI models derived from the basic model (V2.1.0, V2.2.0), the version numbers of the third-generation AI models derived from the basic model (V2.1.1, V2.1.2), and the version number of the fourth-generation AI model derived from the basic model (V2.1.1.1). For example, the AI model with version number V2.1.1 and the AI model with version number V2.1.2 were both obtained by training the AI model with version number V2.1.0.
[0034] Based on the version number of the AI model, the version chain of the AI model, and the preset version number encoding rules, determine the version number of the trained AI model. For example, Figure 2 The version chain in the example provides a pre-defined version number encoding rule. The version number of the base model is V1.0.0, the version number of the AI model obtained by training the base model for the first time is V2.0.0, the version number of the AI model obtained by training the base model for the second time is V3.0.0, the version number of the AI model obtained by training the base model for the third time is V4.0.0, and so on. That is, the version number of the first generation AI model is obtained by increasing the value of the first digit of the base model's version number according to the number of training sessions. The version number of the AI model obtained by training the AI model with version number V2.0.0 for the first time is V2.1.0, the version number of the AI model obtained by training the AI model with version number V2.0.0 for the second time is V2.2.0, and so on. That is, the version number of the second generation AI model is obtained by increasing the value of the first digit of the base model's version number according to the number of training sessions. The version number of the third-generation AI model is obtained by successively increasing the value of the second digit of the version number of the second-generation model based on the number of training iterations. For example, the version number of the AI model with version number V2.1.0 after the first training iteration is V2.1.1, the version number of the AI model with version number V2.1.0 after the second training iteration is V2.1.1, and so on. That is, the version number of the third-generation AI model is obtained by successively increasing the value of the third digit of the version number of the second-generation model based on the number of training iterations. It should be noted that if the version number of the previous generation is not long enough, one digit needs to be added. For example, the version number of the AI model with version number V2.1.1 after the first training iteration is V2.1.1.1, and so on. That is, the version number of the fourth-generation AI model is obtained by adding one digit to the version number of the second-generation model, and then successively increasing the value of that digit based on the number of training iterations.
[0035] In the metadata of the above training task instruction, replace the version number of the above AI model with the version number of the above trained AI model to obtain the metadata of the above trained AI model, and record the version number of the above trained AI model in the version chain of the above AI model. The version chain of the aforementioned AI model is bound to the name of the aforementioned AI model and then stored in the aforementioned fourth database.
[0036] For example, since AI models derived from the same base model have the same name and the same version chain, the version chain of the base model needs to be updated each time a derived model of the base model is trained.
[0037] In an optional embodiment, after delivering and validating the AI model described above, the method further includes: Obtain the name of the AI model from the metadata in the aforementioned delivery task instruction; Based on the name of the AI model, obtain the version chain of the AI model from the fourth database. When the above delivery verification passes, a delivery verification pass flag is set for the version number of the AI model in the version chain of the AI model. When the above delivery verification fails, a delivery verification fail flag is set for the version number of the AI model in the version chain of the AI model.
[0038] For example, when delivery verification fails, a delivery verification failure flag is set for the version number of the AI model in the version chain, so that if the service is in an abnormal running state in the future, it can be quickly determined that the version number can be rolled back.
[0039] In an optional embodiment, after deploying the above services to the target device, the method further includes: Regularly check whether the above services are in an abnormal operating state; For example, the error rate of the service's API (Application Programming Interface) can be checked to see if it is greater than 5% to determine if the service is in an abnormal operating state. If it is greater than 5%, the service is in an abnormal operating state; otherwise, the service is in a normal operating state. Alternatively, other methods can be used to check if the service is in an abnormal operating state. This application does not impose any restrictions on such methods.
[0040] If the above-mentioned service is in the above-mentioned abnormal operating state, obtain the name of the AI model corresponding to the above-mentioned service and the version number of the AI model corresponding to the above-mentioned service from the metadata in the above-mentioned deployment instructions; Based on the name of the AI model corresponding to the above service, retrieve the version chain of the AI model corresponding to the above service from the fourth database; Based on the version number of the AI model and the version chain of the AI model, the rollback version number is determined. The rollback version number is the first version number in the version chain of the AI model that has the delivery verification passed identifier along the version backtracking direction of the version number of the AI model. For example, Figure 3 The example shows a portion of the version chain of an AI model with version number V2.1.1, where the direction of the dashed arrows indicates the version backtracking direction of this AI model. Verify delivery using the identifier. Since the delivery verification failed identifier is used, the rollback version number should be v2.0.0. It should be noted that the rollback version number is the first version number with the delivery verification passed identifier in the version chain of the AI model along the version backtracking direction of the AI model's version number. It refers to the first version number with the delivery verification passed identifier along the version backtracking direction of the AI model's version number, excluding the version number of the AI model itself.
[0041] Based on the name of the AI model and the rollback version number, retrieve the corresponding service from the third database. Deploy the above services to the above target devices.
[0042] For example, when an abnormal operating state is detected in the service corresponding to an AI model, a stable AI model that can be rolled back can be quickly identified through the version chain of the AI model, resulting in a rapid response and thus improving the stability and reliability of the AI model's operation.
[0043] In one optional embodiment, the above-mentioned AI model is trained to obtain a trained AI model, and the trained AI model is then tested and verified, which can be achieved as follows: The trained AI model was tested and validated to obtain multiple test validation metrics, including at least: test accuracy metrics, inference latency metrics, memory usage metrics, and robustness metrics. For example, the test accuracy index represents the accuracy of the trained AI model in performing inference tasks, the inference latency index represents the time required for the trained AI model to complete one inference task, the memory usage index represents the memory space required for the trained AI model to complete one inference task, and the robustness index represents the accuracy of the trained AI model in performing inference tasks in adversarial example scenarios, reflecting the AI model's ability to resist input perturbations. Adversarial examples are designed input data that can easily cause the AI model to make incorrect predictions or decisions.
[0044] If at least one of the above-mentioned test verification indicators meets the corresponding test verification indicator conditions, the above test verification is determined to be passed.
[0045] For example, the test verification criteria for the test accuracy metric is that the test accuracy metric is greater than or equal to 95%. If this condition is met, the test accuracy metric meets the test verification criteria. The test verification criteria for the inference latency metric is that the inference latency metric is less than or equal to 50ms. If this condition is met, the inference latency metric meets the test verification criteria. The test verification criteria for the memory usage metric is that the memory usage metric is ≤2GB. If this condition is met, the memory usage metric meets the test verification criteria. The test verification criteria for the robustness metric is that the robustness metric is greater than or equal to 95%. If this condition is met, the robustness metric meets the test verification criteria.
[0046] In one optional embodiment, the AI model is delivered and verified. Upon successful delivery verification, the service corresponding to the AI model can be obtained as follows: In the shadow environment, the AI model is tested using simulated production traffic, and multiple business verification indicators are obtained. These multiple business verification indicators include at least: throughput indicators and stability indicators. For example, a shadow environment is a virtual test environment that runs in parallel with the production environment to verify the performance and stability of the AI model. It simulates the input and traffic patterns of inference task requests that are highly similar to those in the production environment. The throughput metric represents the number of inference tasks that the AI model can successfully complete per unit time. The stability metric is used to characterize the reliability of the AI model over a long period of time. The stability metric can be the error rate of the AI model in performing inference tasks recorded in a predetermined number of consecutive tests.
[0047] Obtain the manual verification results of the above AI model. The manual verification results are based at least on business alignment, data deviation and compliance review. For example, the AI model is manually reviewed to determine whether it meets the actual business logic requirements, to verify whether the business alignment of the AI model is qualified; the deviation between the data used to train the AI model and the data in the production environment is reasonable, to verify whether the data deviation of the AI model is qualified; and the AI model complies with relevant laws and regulations, to verify whether the compliance of the AI model is qualified. If the business alignment, data deviation, and compliance of the AI model are all qualified, the manual verification result of the AI model is passed; otherwise, it is failed.
[0048] The AI model is delivered by encapsulating it into a service and verifying the interface of the service. For example, the above service is validated via interface verification to verify whether the service executes the inference task according to the design logic and whether the output results meet expectations.
[0049] If all the above-mentioned business verification indicators meet the corresponding business verification indicator conditions, the above-mentioned manual verification result is passed, and the above-mentioned interface verification is passed, then the above-mentioned delivery verification is determined to be passed, and the above-mentioned service is obtained.
[0050] For example, the business verification condition for the throughput indicator is that it is greater than a predetermined quantity, and the business verification condition for the throughput indicator is that the throughput indicator is greater than the predetermined quantity. The business verification condition for the stability indicator is that it is less than a preset error rate, and the business verification condition for the stability indicator is that the stability indicator is less than the preset error rate.
[0051] After binding the AI model and its metadata to the first database, the method further includes: The aforementioned AI model and its metadata are backed up and stored in the object storage service system.
[0052] For example, the AI model and its metadata are backed up and stored in an Object Storage Service (OSS) system to prevent data loss.
[0053] Figure 4 An AI model management device provided in this application embodiment, such as Figure 4 As shown, the above-mentioned device includes: The model management unit 200 is used to bind the AI model and its metadata to the first database. The AI model is the basic model, and the metadata includes at least: name and version number. The training unit 300 is used to respond to the metadata in the training task instruction, select the corresponding AI model from the first database, train the AI model to obtain the trained AI model, test and verify the trained AI model, and when the test and verification are passed, bind the metadata of the trained AI model and the trained AI model and store them in the second database. The algorithm delivery unit 400 is used to respond to the metadata in the delivery task instruction, select the corresponding AI model from the second database, perform delivery verification on the AI model, obtain the service corresponding to the AI model when the delivery verification is passed, and store the metadata of the AI model and the service bound to the third database. The service registration unit 500 is used to select the corresponding service from the third database in response to the metadata in the deployment instruction and deploy the service to the target device.
[0054] Through the above embodiments, in the development environment, the trained AI model is tested and verified. When the test and verification pass, the metadata of the trained AI model and the trained AI model itself are bound and stored in a second database. In response to the metadata in the delivery task instruction, the corresponding AI model is selected from the second database, and the AI model is delivered and verified. When the delivery verification passes, the service corresponding to the AI model is obtained, and the metadata of the AI model and the service are bound and stored in a third database. In the production environment, in response to the metadata in the deployment instruction, the corresponding service is selected from the third database, and the service is deployed to the target device. That is, the second database only stores AI models that have passed the test and verification, and the third database only stores services corresponding to AI models that have passed both the test and verification and the delivery verification. Since the corresponding service is selected from the third database in response to the metadata in the deployment instruction in the production environment, the service deployed to the target device must be the service corresponding to an AI model that has passed both the test and verification and the delivery verification. In other words, the service deployed to the target device must be the service corresponding to a fully verified AI model. This solves the problem that existing AI model management methods are prone to misdeploying AI models that have not yet passed sufficient verification.
[0055] Upon successful completion of the above tests, after binding and storing the metadata of the trained AI model and the trained AI model itself in the second database, the training unit is further used for: The metadata of the trained AI model and the trained AI model itself are bound together and stored in the first database.
[0056] Upon successful verification of the above tests, before storing the metadata of the trained AI model and the trained AI model itself in the second database, the training unit is also used for: Obtain the name and version number of the AI model from the metadata in the training task instructions. Based on the name of the AI model, the version chain of the AI model is obtained from the fourth database. The version chain is used to record the genealogical relationship and version number of the AI models derived from the base model of the AI model. Based on the version number of the AI model, the version chain of the AI model, and the preset version number encoding rules, determine the version number of the trained AI model. In the metadata of the above training task instruction, replace the version number of the above AI model with the version number of the above trained AI model to obtain the metadata of the above trained AI model, and record the version number of the above trained AI model in the version chain of the above AI model. The version chain of the aforementioned AI model is bound to the name of the aforementioned AI model and then stored in the aforementioned fourth database.
[0057] In an optional embodiment, after the AI model is delivered and verified, the algorithm delivery unit is further configured to: Obtain the name of the AI model from the metadata in the aforementioned delivery task instruction; Based on the name of the AI model, obtain the version chain of the AI model from the fourth database. When the above delivery verification passes, a delivery verification pass flag is set for the version number of the AI model in the version chain of the AI model. When the above delivery verification fails, a delivery verification fail flag is set for the version number of the AI model in the version chain of the AI model.
[0058] In an optional embodiment, after the service is deployed to the target device, the service registration unit is further configured to: Regularly check whether the above services are in an abnormal operating state; If the above-mentioned service is in the above-mentioned abnormal operating state, obtain the name of the AI model corresponding to the above-mentioned service and the version number of the AI model corresponding to the above-mentioned service from the metadata in the above-mentioned deployment instructions; Based on the name of the AI model corresponding to the above service, retrieve the version chain of the AI model corresponding to the above service from the fourth database; Based on the version number of the AI model and the version chain of the AI model, the rollback version number is determined. The rollback version number is the first version number in the version chain of the AI model that has the delivery verification passed identifier along the version backtracking direction of the version number of the AI model. Based on the name of the AI model and the rollback version number, retrieve the corresponding service from the third database. Deploy the above services to the above target devices.
[0059] In one optional embodiment, the above-mentioned AI model is trained to obtain a trained AI model, and the trained AI model is then tested and verified, which can be achieved as follows: The trained AI model was tested and validated to obtain multiple test validation metrics, including at least: test accuracy metrics, inference latency metrics, memory usage metrics, and robustness metrics. If at least one of the above-mentioned test verification indicators meets the corresponding test verification indicator conditions, the above test verification is determined to be passed.
[0060] In one optional embodiment, the AI model is delivered and verified. Upon successful delivery verification, the service corresponding to the AI model can be obtained as follows: In the shadow environment, the AI model is tested using simulated production traffic, and multiple business verification indicators are obtained. These multiple business verification indicators include at least: throughput indicators and stability indicators. Obtain the manual verification results of the above AI model. The manual verification results are based at least on business alignment, data deviation and compliance review. The AI model is delivered by encapsulating it into a service and verifying the interface of the service. If all the above-mentioned business verification indicators meet the corresponding business verification indicator conditions, the above-mentioned manual verification result is passed, and the above-mentioned interface verification is passed, then the above-mentioned delivery verification is determined to be passed, and the above-mentioned service is obtained.
[0061] In an optional embodiment, after binding the AI model and its metadata to the first database, the model association unit is further configured to: The aforementioned AI model and its metadata are backed up and stored in the object storage service system.
[0062] This application also provides an AI model management system, such as Figure 5 As shown, the AI model management system includes: a first database 10, a second database 20, a third database 30, and a fourth database 40, and a management module 60. The management module 60 is used to implement the steps of any of the above-described AI model management methods.
[0063] For example, such as Figure 5 As shown, the AI model management system also includes: an object storage service system 50.
[0064] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the AI model management method provided in any of the foregoing method embodiments.
[0065] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0066] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0067] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.
[0068] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.
Claims
1. An AI model management method, characterized in that, The method includes: The AI model and its metadata are bound together and stored in the first database. The AI model is the base model, and the metadata includes at least: name and version number. In response to the metadata in the training task instruction, a corresponding AI model is selected from the first database, the AI model is trained to obtain a trained AI model, the trained AI model is tested and verified, and when the test and verification are passed, the metadata of the trained AI model and the trained AI model are bound and stored in the second database. In response to the metadata in the delivery task instruction, the corresponding AI model is selected from the second database, the AI model is delivered and verified, the service corresponding to the AI model is obtained when the delivery verification is passed, and the metadata of the AI model and the service are bound and stored in the third database. In response to the metadata in the deployment instruction, the corresponding service is selected from the third database and deployed to the target device.
2. The method according to claim 1, characterized in that, Upon successful test verification, after binding and storing the metadata of the trained AI model and the trained AI model in the second database, the method further includes: The metadata of the trained AI model and the trained AI model itself are bound together and stored in the first database.
3. The method according to claim 1 or 2, characterized in that, Upon successful test verification, before binding and storing the metadata of the trained AI model and the trained AI model in the second database, the method further includes: Obtain the name and version number of the AI model from the metadata in the training task instructions; The version chain of the AI model is obtained from the fourth database based on the name of the AI model. The version chain is used to record the genealogical relationship and version number of AI models derived from the base model of the AI model. The version number of the trained AI model is determined based on the version number of the AI model, the version chain of the AI model, and the preset version number encoding rules. In the metadata of the training task instruction, the version number of the AI model is replaced with the version number of the trained AI model to obtain the metadata of the trained AI model, and the version number of the trained AI model is recorded in the version chain of the AI model. The version chain of the AI model is bound to the name of the AI model and then stored in the fourth database.
4. The method according to claim 3, characterized in that, After delivering and validating the AI model, the method further includes: Obtain the name of the AI model from the metadata in the delivery task instruction; Based on the name of the AI model, obtain the version chain of the AI model from the fourth database; When the delivery verification passes, a delivery verification pass flag is set for the version number of the AI model in the version chain; when the delivery verification fails, a delivery verification fail flag is set for the version number of the AI model in the version chain.
5. The method according to claim 4, characterized in that, After deploying the service to the target device, the method includes: Periodically check whether the service is in an abnormal operating state; When the service is in the abnormal running state, the name of the AI model corresponding to the service and the version number of the AI model corresponding to the service are obtained from the metadata in the deployment instruction; Based on the name of the AI model corresponding to the service, the version chain of the AI model corresponding to the service is obtained from the fourth database; Based on the version number of the AI model and the version chain of the AI model, a rollback version number is determined. The rollback version number is the first version number of the AI model in the version chain that has the delivery verification passed identifier along the version backtracking direction. Based on the name of the AI model and the rollback version number, the corresponding service is obtained from the third database; Deploy the service to the target device.
6. The method according to claim 1, characterized in that, The trained AI model is tested and validated, including: The trained AI model is tested and verified to obtain multiple test and verification metrics, including at least: test accuracy metrics, inference latency metrics, memory usage metrics, and robustness metrics. The test verification is deemed successful if at least one of the test verification indicators meets the corresponding test verification indicator conditions.
7. The method according to claim 1, characterized in that, The AI model is delivered and validated. Upon successful delivery validation, the corresponding service for the AI model is obtained, including: In the shadow environment, the AI model is tested using simulated production traffic, and multiple business verification indicators are obtained. These multiple business verification indicators include at least: throughput indicators and stability indicators. Obtain the manual verification results of the AI model, which are based at least on business alignment, data deviation, and compliance review. The AI model is delivered by encapsulating it into a service and verifying the interface of the service. If all the business verification indicators meet the corresponding business verification indicator conditions, the manual verification result is passed, and the interface verification is passed, then the delivery verification is determined to be passed, and the service is obtained.
8. The method according to claim 1, characterized in that, After binding the AI model and its metadata to the first database, the method includes: The AI model and its metadata are backed up and stored in the object storage service system.
9. An AI model management device, characterized in that, The device includes: The model management unit is used to bind and store the AI model and its metadata in the first database. The AI model is a basic model, and the metadata includes at least: name and version number. The training unit is used to respond to the metadata in the training task instruction, select the corresponding AI model from the first database, train the AI model to obtain the trained AI model, test and verify the trained AI model, and when the test and verification are passed, bind the metadata of the trained AI model and the trained AI model and store them in the second database. The algorithm delivery unit is used to respond to the metadata in the delivery task instruction, select the corresponding AI model from the second database, perform delivery verification on the AI model, obtain the service corresponding to the AI model when the delivery verification is passed, and store the metadata of the AI model and the service bound to the third database. The service registration unit is used to select the corresponding service from the third database in response to the metadata in the deployment instruction and deploy the service to the target device.
10. An AI model management system, characterized in that, The AI model management system includes: The system comprises a first database, a second database, a third database, and a fourth database, and a management module, wherein the management module is used to implement the steps of the AI model management method as described in any one of claims 1 to 8.