System for supporting dynamic release of model

Abstract

The invention provides a system for supporting dynamic release of a model, and the system is located at a cloud end and comprises six functional modules: a model online module, a model warehouse, a model store, a model transaction module, a model updating module and a model monitoring module, and the output of the model online module is connected with the input of the model warehouse. The output of the model warehouse is connected with the input of the model store and the model updating module, the output of the model store is connected with the input of the model transaction module, the model transaction module is connected with the model warehouse, and the output of the model updating module is connected with the input of the model online module. The model transaction module, the model updating module and the model monitoring module are connected with the user cluster of the edge end, and the input of the model monitoring module is connected with the output of all the modules. The system can automatically complete verification, release and update processes of the model, realizes real-time performance and safety of model release, can improve model release and deployment efficiency, promotes rapid iteration of the model, and enhances usability of the system.

Description

Technical Field

[0001] The invention relates to the field of intelligent model publishing. Background Art

[0002] With the rapid development of information technology, emerging application scenarios such as the Industrial Internet are emerging one after another. These scenarios have put forward higher requirements on the ability to accurately perceive data and respond in real time. At the same time, the field of artificial intelligence has made significant progress in recent years, and the deployment of intelligent models at the edge has gradually become the focus of the industry. More and more application scenarios require the use of intelligent models. As a key link between intelligent model development and users, the efficiency and flexibility of model publishing directly affect the competitiveness of the model.

[0003] Existing model publishing technologies take into account the differences in the types of different target deployment systems, which simplifies the model publishing process to a certain extent. However, existing publishing technologies only consider the single release of the model and lack the ability to dynamically adjust. This means that after the system model is determined, any changes need to go through a cumbersome re-release process, which is not only time-consuming and labor-intensive, but may also introduce new risks. In addition, existing technologies lack intelligent support in the release process and often rely on manual operations. The process is cumbersome, inflexible, prone to human errors, and difficult to ensure the consistency and traceability of the release. When faced with large-scale systems and complex dependencies, manual operations are often inefficient and difficult to meet the needs of rapid iteration and real-time response. Summary of the invention

[0004] In order to solve the problem of cumbersome and poor flexibility in the intelligent model publishing process, the present invention proposes a system that supports dynamic model publishing. The system can automatically complete the model verification, publishing and updating process, achieve the real-time and security of model publishing, improve the efficiency of model publishing and deployment, and promote rapid model iteration.

[0005] Technical solution of the present invention: A system supporting dynamic model publishing, the system is located in the cloud, and includes six functional modules: a model online module, a model warehouse, a model store, a model transaction module, a model update module, and a model monitoring module. The output of the model online module is connected to the input of the model warehouse, the output of the model warehouse is connected to the input of the model store and the model update module, the output of the model store is connected to the input of the model transaction module, the model transaction module is connected to the model warehouse, the output of the model update module is connected to the input of the model online module, the model transaction module, the model update module, and the model monitoring module are connected to the user cluster at the edge, and the input of the model monitoring module is connected to the output of all modules; The model online module is used for uploading, testing and publishing the model; The model warehouse is used to store and manage the model files and metadata uploaded by the model developers in the model online module, and supports version control, tag management, and search query functions. The model warehouse stores the model and its basic information in the form of container images; The model store displays the published models and their basic information in the model warehouse in the form of a list, and provides model subscription and deployment services to model users through the model transaction module; The model trading module is used to trade models; The model updating module processes the model updating; The model monitoring module monitors the operating status of the model and the entire model publishing system in real time throughout the life cycle of the system.

[0006] The present invention proposes a system that supports dynamic model publishing. The system can automatically complete the model verification, publishing and updating process, realize the real-time and security of model publishing, improve the efficiency of model publishing and deployment, promote rapid model iteration and enhance system usability.

[0007] The present invention has the following advantages and beneficial effects: (1) The present invention supports dynamic publishing of various types of models. When the model is adjusted, the system can respond in real time to complete the update of the publishing, which simplifies the publishing process and improves the real-time performance of the system. (2) The present invention uses technologies such as Kubernetes and CI / CD to realize automatic model release. Developers only need to focus on model development without paying attention to the specific process of model release, which reduces the development difficulty and improves the efficiency of model release. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 A schematic diagram of a system framework supporting dynamic model publishing of the present invention.

[0009] Figure 2 The figure is a schematic diagram of the workflow of the model online module in the system of the present invention.

[0010] Figure 3 The figure is a schematic diagram of the workflow of the model trading module in the system of the present invention.

[0011] Figure 4 Schematic diagram of the workflow of the model updating module in the system of the present invention.

[0012] Figure 5 The figure is a schematic diagram of the working process of the model monitoring module in the system of the present invention. Specific implementation plan The technical solution provided by the present application will be further described below in conjunction with specific embodiments and accompanying drawings. The advantages and features of the present application will become more apparent with the following description.

[0014] like Figure 1 As shown, a system supporting dynamic model publishing is located in the cloud and includes six functional modules: a model launch module, a model warehouse, a model store, a model transaction module, a model update module, and a model monitoring module. The output of the model launch module is connected to the input of the model warehouse, the output of the model warehouse is connected to the input of the model store and the model update module, the output of the model store is connected to the input of the model transaction module, the model transaction module is connected to the model warehouse, the output of the model update module is connected to the input of the model launch module, the model transaction module, the model update module, and the model monitoring module are connected to the user cluster at the edge, and the input of the model monitoring module is connected to the output of all modules; wherein: The model launch module includes a model upload submodule, a model test submodule, and a model release submodule, wherein: the model upload submodule is used to collect model files uploaded by model developers and their basic information, and the basic information includes the model version, introduction, API description, pricing, dependent models, test cases, and dependent environments; the model test submodule is used to complete the verification test of the model function; the model release submodule is used to release the verified model, and can customize the model release solution according to the needs of different models; The model warehouse is used to store and manage the model files and metadata uploaded by the model developers in the model launch module, and supports version control, tag management, search query and other functions. The model warehouse stores the model in the form of container images, and stores the model version, introduction, API description, pricing, subscription records and other information; The model store displays the published models in the model warehouse in the form of a list, including the version, introduction, API description, pricing, and subscription record of the model, and provides model subscription and deployment services to model users through the model transaction module; The model transaction module is used to trade models, and includes a subscription model submodule, a permission allocation submodule, a subscription information storage submodule, and a model deployment submodule, wherein: the subscription model submodule is used to receive model subscription requests from model users and determine the models and APIs required by users; the permission allocation submodule is used by developers to allocate model deployment and API usage permissions to users; the subscription information storage submodule updates the model subscription records in the model warehouse and notifies the model developer of the model subscription status; the model deployment submodule allows users to deploy models with permissions in their own edge clusters; The model update module includes a submit update submodule, an original version model storage submodule, a system model update submodule, and a user model update submodule, wherein: the submit update submodule is used to process the model update request submitted by the developer and obtain the updated model information; the original version model storage submodule is used to store the original version model information; the system model update submodule inputs the updated model information into the model online module; the user model update submodule completes the model update according to the user's needs; The model monitoring module includes a system detection submodule, an alarm submodule, an abnormal control submodule, and a model performance monitoring submodule, wherein: the system detection submodule is used to monitor the operating status of the entire cloud model dynamic publishing system and the deployed model on the user side in real time; the model performance monitoring submodule is used to monitor performance indicators, resource utilization, abnormal status and other information in real time; the abnormal control submodule and the alarm submodule are used to complete control measures and automatically trigger the alarm mechanism when abnormal situations are discovered or performance thresholds are reached, and notify the system administrator for processing.

[0015] The system mainly includes three types of users: system administrators, model developers, and model users; among them, The system administrator has the highest authority and can manage the entire cloud model dynamic publishing system, including six modules: model online module, model warehouse, model store, model transaction module, model update module, and model monitoring module. It has functions such as system maintenance, model management, and user management. The model developer exists in the upload model submodule of the model launch module, the submit update submodule of the model update module, and the permission allocation submodule of the model transaction module, and can upload, update, and allocate permissions to the model, wherein: when uploading the model, the model developer uploads the model file and its basic information to the upload model submodule of the model launch module, and the basic information includes the model version, introduction, API description, pricing, dependent model, test case, and dependent environment; when updating the model, the model developer updates the model according to the model application requirements and performance requirements, and resubmits the model file and its basic information in the submit update submodule of the model update module; when allocating permissions, the developer allocates model deployment and API usage permissions according to the model requirements of the model user in the permission allocation submodule of the model transaction module; The model user exists in the subscription model submodule and model deployment submodule of the model transaction module, and the user model update submodule of the model update module, and can subscribe to and deploy models, wherein: in the subscription model submodule of the model transaction module, the model user accesses the published models in the model store, views the model information, and subscribes to the models and APIs that meet the requirements according to the usage requirements; in the model deployment submodule of the model transaction module, the model user can deploy the subscribed models; in the user model update submodule of the model update module, the model user can view the updates of the model and complete the model updates on the user side according to the requirements.

[0016] like Figure 2 As shown, in the model online module, the uploading, testing and publishing of the model specifically include the following steps: (1) When uploading the model submodule, the developer submits the model file and its basic information. The basic information includes the model version, introduction, API description, pricing, dependent models, test cases, dependent environment, etc. The API description refers to the API that the model provides for model users to call. The API is determined by the developer based on the model type and interaction requirements. Pricing refers to the price of the model, which is the remuneration paid by the user when using the model. The dependent model refers to the basic model required for the model to run. The dependent environment refers to the dependent environment when the model is deployed. To facilitate the deployment of the model, the model file and its dependent environment are packaged into a container image through the pre-written Dockerfile file.

[0017] (2) The uploaded model is further deployed and tested through the model testing submodule. The system enters the automated process, triggers the deployment process through the CI / CD tool, and calls the container orchestration tool Kubernetes to deploy and test the packaged model container image. Before deployment, it is necessary to traverse the model's dependent models. If the model has a dependent basic model, the basic model needs to be deployed. CI / CD is the abbreviation of continuous integration (CI), continuous delivery (CD), and continuous deployment (CD).

[0018] (3) After the model is deployed in the model testing submodule, the system automatically performs model verification tests based on the test cases submitted by the developer, including functional tests and performance tests, to ensure that the model can run normally and function correctly; Functional testing verifies the performance of the model in actual business scenarios, simulating user requests based on test cases submitted by developers and verifying the correctness of model responses; Performance testing evaluates the model's resource usage, response speed, and other performance characteristics; In addition, after a model update or system upgrade, regression testing is automatically performed to ensure that the new version of the model or system does not introduce new problems.

[0019] (4) After the model passes the test, the model is marked as online through the model release submodule, and the model file and its basic information are stored in the model warehouse.

[0020] like Figure 3 As shown, in the model transaction module, the transaction of the model includes the following steps: (1) Users can view the models currently published in the model store through the subscription model submodule, select the required models and API subscriptions based on service requirements, and complete the payment according to the pricing set by the developer; (2) After obtaining the user's subscription model request through the permission allocation submodule, the developer allocates model deployment permissions and API usage permissions based on the content paid by the user; (3) Update the model subscription information in the model repository through the subscription information storage submodule and notify the model developer; (4) Through the model deployment submodule, the user completes the configuration of the model, including the model version, deployment location, etc. The system will deploy the model on the user side. If the model has dependencies, the dependent basic model will be deployed from the model repository first based on the dependent model information provided by the developer, and a data exchange method between models will be established.

[0021] like Figure 4 As shown, in the model updating module, the model updating specifically includes the following steps: (1) By submitting an updated sub-module, the developer updates and optimizes the model according to the model application requirements and performance requirements, and submits the updated model file and its basic model information; (2) The original version model storage submodule stores the original model and uses the version control system Git to perform version control on the model file; (3) The new model is input into the model testing submodule in the model online module through the system model update submodule, and the model testing submodule completes the model testing, further completes the release of the new model and updates the model in the model warehouse; (4) Access the original model subscription record in the model repository through the user model update submodule, and issue model update reminders to all users of this model. Users can view the updates and choose to keep the original version or visit the model store to complete the model upgrade.

[0022] like Figure 5 As shown in the figure, the model monitoring module monitors the running status of the model and the entire model publishing system in real time throughout the life cycle of the system. The model monitoring specifically includes the following steps: (1) The system monitoring submodule records and analyzes the performance of the entire system and the user-side deployment model in real time. The model performance monitoring submodule records and analyzes the runtime performance indicators in real time, including response time, throughput, latency, and resource utilization, and stores them in logs. (2) When an abnormal situation is detected or the preset performance indicator threshold is reached, the abnormal control submodule uses an intelligent algorithm to automatically identify and classify the fault signal, and executes the corresponding control task through a pre-written automated script. If it is an unidentifiable problem, the alarm submodule will trigger the alarm mechanism, notify the system administrator, and force the use of the model to stop.

[0023] The above description is only a description of the preferred embodiments of the present application, and is not intended to limit the scope of the present application. Any changes or modifications made by any person skilled in the art based on the above disclosed technical contents shall be deemed as equivalent effective embodiments and shall fall within the scope of protection of the technical solution of the present application.

Claims

1. A system supporting dynamic model publishing, characterized in that: The system is located in the cloud and includes six functional modules: model online module, model warehouse, model store, model transaction module, model update module, and model monitoring module. The output of the model online module is connected to the input of the model warehouse, the output of the model warehouse is connected to the input of the model store and the model update module, the output of the model store is connected to the input of the model transaction module, the model transaction module is connected to the model warehouse, the output of the model update module is connected to the input of the model online module, the model transaction module, the model update module, and the model monitoring module are connected to the user cluster at the edge, and the input of the model monitoring module is connected to the output of all modules; The model online module is used for uploading, testing and publishing the model; The model warehouse is used to store and manage the model files and metadata uploaded by the model developers in the model launch module, and supports version control, tag management, and search query functions. The model warehouse stores the model and its basic information in the form of container images; The model store displays the published models and their basic information in the model warehouse in the form of a list, and provides model subscription and deployment services to model users through the model transaction module; The model trading module is used to trade models; The model updating module processes model updating; The model monitoring module monitors the operating status of the model and the entire model publishing system in real time throughout the life cycle of the system.

2. A system for supporting dynamic model publishing as claimed in claim 1, characterized in that: The model online module includes a model upload submodule, a model test submodule, and a model release submodule, wherein: the model upload submodule is used to collect model files and basic information uploaded by model developers; the model test submodule is used to complete the verification test of model functions; the model release submodule is used to release the verified model and can customize the model release solution according to the needs of different models; The model transaction module includes a subscription model submodule, a permission allocation submodule, a subscription information storage submodule, and a model deployment submodule, wherein: the subscription model submodule is used to receive model subscription requests from model users and determine the models and APIs required by users; the permission allocation submodule is used by developers to allocate model deployment and API usage permissions to users; the subscription information storage submodule updates the model subscription records in the model warehouse and notifies the model developer of the model subscription status; the model deployment submodule allows users to deploy models with permissions in their own edge clusters; The model update module includes a submit update submodule, an original version model storage submodule, a system model update submodule, and a user model update submodule, wherein: the submit update submodule is used to process the model update request submitted by the developer and obtain the updated model information; the original version model storage submodule is used to store the original version model information; the system model update submodule inputs the updated model information into the model online module; the user model update submodule completes the model update according to the user's needs; The model monitoring module includes a system detection submodule, an alarm submodule, an abnormal control submodule, and a model performance monitoring submodule, wherein: the system detection submodule is used to monitor the operating status of the entire cloud model dynamic publishing system and the deployed model on the user side in real time; the model performance monitoring submodule is used to monitor performance indicators, resource utilization, abnormal status and other information in real time; the abnormal control submodule and the alarm submodule are used to complete control measures and automatically trigger the alarm mechanism when abnormal situations are discovered or performance thresholds are reached, and notify the system administrator for processing.

3. A system for supporting dynamic model publishing as claimed in claim 2, characterized in that: The system mainly includes three types of users: system administrators, model developers, and model users; among them, The system administrator has the highest authority and can manage the entire cloud model dynamic publishing system, including six modules: model online module, model warehouse, model store, model transaction module, model update module, and model monitoring module. It has functions such as system maintenance, model management, and user management. The model developer exists in the upload model submodule of the model online module, the submit update submodule of the model update module, and the permission allocation submodule of the model transaction module, and can upload, update and allocate permissions to the model, wherein: when uploading the model, the model developer uploads the model file and its basic information to the upload model submodule of the model online module; when updating the model, the model developer updates the model according to the model application requirements and performance requirements, and resubmits the model file and its basic information in the submit update submodule of the model update module; when allocating permissions, the developer allocates model deployment and API usage permissions according to the model requirements of the model user in the permission allocation submodule of the model transaction module; The model user exists in the subscription model submodule and model deployment submodule of the model transaction module, and the user model update submodule of the model update module, and can subscribe to and deploy models, wherein: in the subscription model submodule of the model transaction module, the model user accesses the published models in the model store, views the model information, and subscribes to the models and APIs that meet the requirements according to the usage requirements; in the model deployment submodule of the model transaction module, the model user can deploy the subscribed models; in the user model update submodule of the model update module, the model user can view the updates of the model and complete the model updates on the user side according to the requirements.

4. A system for supporting dynamic model publishing as claimed in claim 2, characterized in that: In the model online module, the uploading, testing and publishing of the model specifically include the following steps: (1) When uploading a model submodule, the developer submits the model file and its basic information. The basic information includes the model version, introduction, API description, pricing, dependent models, test cases, dependent environment, etc. The API description refers to the API that the model provides for model users to call. The API is determined by the developer based on the model type and interaction requirements. Pricing refers to the price of the model, which is the remuneration paid by the user when using the model. The dependent model refers to the basic model required for the model to run. The dependent environment refers to the dependent environment when the model is deployed. (2) The uploaded model is further deployed and tested through the model testing submodule. The system enters the automated process, triggers the deployment process through the CI / CD tool, and calls the container orchestration tool Kubernetes to deploy and test the packaged model container image. Before deployment, the model's dependent models need to be traversed. If the model has a dependent basic model, the basic model needs to be deployed. CI / CD is the abbreviation of Continuous Integration (CI), Continuous Delivery (CD), and Continuous Deployment (CD). (3) After the model is deployed in the model testing submodule, the system automatically performs model verification tests based on the test cases submitted by the developer, including functional tests and performance tests, to ensure that the model can run normally and function correctly; Functional testing verifies the performance of the model in actual business scenarios, simulating user requests based on test cases submitted by developers and verifying the correctness of model responses; Performance testing evaluates the model's resource usage, response speed, and other performance characteristics; After a model is updated or a system is upgraded, regression testing is automatically performed to ensure that the new version of the model or system does not introduce new problems; (4) After the model passes the test, the model is marked as online through the model release submodule, and the model file and its basic information are stored in the model warehouse.

5. A system for supporting dynamic model publishing as claimed in claim 2, characterized in that: In the model transaction module, the transaction of the model includes the following steps: (1) Users can view the models currently published in the model store through the subscription model submodule, select the required models and API subscriptions based on service requirements, and complete the payment according to the pricing set by the developer; (2) After obtaining the user's subscription model request through the permission allocation submodule, the developer allocates model deployment permissions and API usage permissions based on the content paid by the user; (3) Update the model subscription information in the model repository through the subscription information storage submodule and notify the model developer; (4) Through the model deployment submodule, the user completes the configuration of the model, and the system will deploy the model on the user side. If the model has dependencies, the dependent basic model will be deployed from the model repository first based on the dependent model information provided by the developer, and a data exchange method between models will be established.

6. A system for supporting dynamic model publishing as claimed in claim 2, characterized in that: In the model updating module, model updating specifically includes the following steps: (1) By submitting an updated sub-module, the developer updates and optimizes the model according to the model application requirements and performance requirements, and submits the updated model file and its basic model information; (2) The original version model storage submodule stores the original model and uses the version control system to perform version control on the model file; (3) The new model is input into the model testing submodule in the model online module through the system model update submodule, and the model testing submodule completes the model testing, further completes the release of the new model and updates the model in the model warehouse; (4) Access the original model subscription record in the model repository through the user model update submodule, and issue model update reminders to all users of this model. Users can view the updates and choose to keep the original version or visit the model store to complete the model upgrade.

7. A system for supporting dynamic model publishing as claimed in claim 2, characterized in that: The model monitoring module monitors the operating status of the model and the entire model publishing system in real time throughout the system life cycle. The model monitoring specifically includes the following steps: (1) The system monitoring submodule records and analyzes the performance of the entire system and the user-side deployment model in real time. The model performance monitoring submodule records and analyzes the runtime performance indicators in real time, including response time, throughput, latency, and resource utilization, and stores them in logs. (2) When an abnormal situation is detected or the preset performance indicator threshold is reached, the abnormal control submodule uses an intelligent algorithm to automatically identify and classify the fault signal, and executes the corresponding control task through a pre-written automated script. If it is an unidentifiable problem, the alarm submodule will trigger the alarm mechanism, notify the system administrator, and force the use of the model to stop.

Images