AI task dynamic deployment and management system based on micro servitization

Abstract

The invention belongs to the technical field of deployment and operation management of an artificial intelligence system, and particularly relates to a micro-servitization-based AI task dynamic deployment and management system, which comprises a task management center platform and a plurality of AI execution nodes communicated with the task management center platform through a network, the task management center platform is used for uniformly planning and managing the AI task execution topology in the whole computing resource pool and issuing an AI task deployment instruction in combination with the existing task load condition and the resource utilization rate; the method comprises the following steps of: before an AI task is deployed, carrying out operation environment and model dependency detection on an AI execution node to ensure deployment compatibility; after deployment, task running states and model version information are collected and stored, and the current execution topology is updated; version management, gray release and dynamic capacity expansion and contraction scheduling are carried out on the task micro-service; the task management center platform is provided with an AI task scheduling module; according to the method, automatic environment detection and dependent installation are supported, the problem of deployment failure caused by AI execution node environment difference is solved, and cross-platform environment consistency is achieved.

Description

Technical Field

[0001] This invention belongs to the field of deployment and operation management technology of artificial intelligence systems, specifically referring to a microservice-based AI task dynamic deployment and management system. Background Technology

[0002] With the widespread application of artificial intelligence technologies such as deep learning, natural language processing, and computer vision, the number, scale, and operational modes of AI models are continuously increasing. When multiple AI models are deployed and run on different hardware environments and regional nodes, problems such as complex dependencies, frequent updates, and resource heterogeneity arise.

[0003] Traditional AI task deployment mainly relies on manual or script-based operation and maintenance methods, including model file copying, inference service installation, environment configuration, and version upgrades. When the number of AI tasks is large, the model types are complex, and the number of nodes is large, manual operation becomes inefficient, uncontrollable, and extremely difficult to support dynamic scheduling, automatic scaling, and fault migration capabilities.

[0004] With the development of containerization and microservice technologies, AI model services are gradually shifting from monolithic deployment to service-oriented and modular approaches. However, most existing solutions lack capabilities such as automated dependency detection, cross-platform consistent deployment, model canary releases, and dynamic resource scheduling. Furthermore, AI task execution often relies on specific hardware, specific operator libraries, and specific driver versions, exhibiting strong dependence on environment matching and resulting in insufficient overall system flexibility. Summary of the Invention

[0005] In order to overcome the shortcomings of the prior art, the present invention provides a digital twin remote operation and maintenance method and system for intelligent manufacturing.

[0006] The technical solution adopted by this invention is as follows: This invention provides a microservice-based AI task dynamic deployment and management system, including a task management center platform and multiple AI execution nodes communicating with it via a network. The task management center platform is used to uniformly plan and manage the AI ​​task execution topology in the entire computing resource pool, and issue AI task deployment instructions based on the existing task load and resource utilization. Before AI task deployment, the platform performs runtime environment and model dependency detection on the AI ​​execution nodes to ensure deployment compatibility. After deployment, it collects and stores task running status, model version information, and updates the current execution topology. It also performs version management, canary release, and dynamic scaling scheduling for task microservices. The task management center platform includes an AI task scheduling module. The AI ​​task scheduling module is connected to the environment dependency detection module, microservice version management module, execution topology management module, and first communication module respectively. The AI ​​execution node is used to receive and execute deployment commands from the task management center platform to become a usable AI microservice instance. The AI ​​execution node uses dependency detection mechanism and image verification mechanism to achieve cross-platform deployment, and uses service health check and inference correctness detection mechanism to verify deployment success. The AI ​​execution node includes a dependency detection execution module, a microservice deployment execution module, and a second communication module.

[0007] Furthermore, the execution topology management module is used to collect the deployment status, resource usage information and node performance indicators of AI microservice instances, and store them in the topology database, presenting a visual deployment topology through a web interface; the database also stores the access credentials and resource limit information of AI execution nodes; The microservice version management module is used to uniformly store and manage AI model versions, inference image versions, and runtime dependency versions, and maintain model and image repositories; it supports adding, deleting, upgrading, rolling back, canary releases, and version consistency verification; and it maintains a version record for each AI execution node for subsequent upgrades and management. The environment dependency detection module is used to automatically detect the running environment of the execution node before the AI ​​task is deployed. Based on the detection results, it extracts the corresponding dependent components from the microservice version management module and distributes them to the execution node for deployment. If the dependency is satisfied, the result is passed to the AI ​​task scheduling module. The AI ​​task scheduling module is used to select execution nodes and issue deployment instructions; after confirming that the dependency detection is passed, it applies for the corresponding model version and inference image version from the microservice version management module and automatically deploys them to the execution node; and adopts a concurrent deployment method to achieve efficient batch scheduling.

[0008] Furthermore, the dependency detection execution module is used to parse dependency detection instructions from the task management center platform, automatically execute environment detection scripts, output dependency detection results, and automatically install dependency component packages issued by the center platform and return the results. The dependency detection description document uses YAML; The microservice deployment and execution module is used to automatically pull model files, inference service images and runtime dependencies, parse deployment description documents and automatically complete microservice and model loading and startup; after deployment, it performs health checks and inference correctness verification, and the verification methods include cold start latency detection, heartbeat packet detection, example inference verification or port availability verification. The second communication module includes a high-speed model transmission channel and a status feedback channel; the high-speed model transmission channel is based on an encrypted transmission protocol, and the status feedback channel uses a lightweight message queue to achieve real-time status feedback.

[0009] Furthermore, the task management center platform runs on a physical server; the AI ​​execution nodes include container instances, virtual machines, physical machines, and edge computing devices.

[0010] An operational method for a microservice-based AI task dynamic deployment and management system mainly includes the following steps: Step 1: The task management center platform analyzes the current resource status based on the execution topology management module, determines the nodes that need to deploy AI model tasks, and sends the selected node information to the environment dependency detection module; Step 2: The environment dependency detection module communicates with the AI ​​execution node through the first communication module to detect the running environment of the AI ​​execution node, and the execution node returns the detection results; based on the detection results, the deployment phase begins; Step 3: The AI ​​task scheduling module connects to qualified AI execution nodes. The AI ​​execution nodes automatically install the model and inference service images through the microservice deployment and execution module, and return to the task management center after the deployment is completed; the task management center records the deployment results. Step 4: The microservice version management module sends a notification and automatically performs dependency upgrades and model hot updates after confirmation.

[0011] Furthermore, the execution topology management module obtains the current node topology, load, and model distribution, selects the target node based on task requirements, and sends the node credentials to the first communication module. The task management center concurrently connects to multiple AI execution nodes through the first communication module, and uses a threaded asynchronous communication mode to achieve concurrent deployment of multiple nodes. The dependency detection execution module automatically detects the AI ​​execution node environment and returns the detection results to the task management center. If the node environment does not meet the dependency requirements, it issues the missing component package and performs automatic installation. The AI ​​task scheduling module issues the model version and inference image version to the target node, and the AI ​​execution node automatically completes the deployment and returns the results to the task management center.

[0012] Furthermore, the concurrent deployment method of the microservice deployment and execution module adopts an event-driven mechanism.

[0013] Furthermore, the dependency detection execution module detects environment description documents and dependency detection scripts; the execution node executes the script to automatically detect GPU / CPU, runtime environment, model dependencies and network ports, and generates a structured report that is sent back to the task management center platform.

[0014] Furthermore, the concurrent deployment process includes: integrity verification of model files and image packages, automatic deployment of inference services and execution of health checks, verification of the correctness of example inference, and registration of the service to the task management center after successful deployment.

[0015] The beneficial effects achieved by the present invention using the above structure are as follows: The present invention provides a microservice-based AI task dynamic deployment and management system, achieving the following beneficial effects: (1) Supports automatic environment detection and dependency installation, solves the deployment failure problem caused by the difference in AI execution node environment, and achieves cross-platform environment consistency.

[0016] (2) Support automatic management and canary release of model, image and dependency versions to ensure that AI service upgrades are controllable and avoid version conflicts.

[0017] (3) Supports concurrent deployment and dynamic scaling of large-scale nodes, significantly improving deployment efficiency and adapting to large-scale AI tasks.

[0018] (4) Supports hot model updates, automatic rollback and health checks to improve system stability and fault tolerance.

[0019] (5) Support unified management and scheduling of heterogeneous computing resources, improve resource utilization and reduce costs. Attached Figure Description

[0020] Figure 1 This is a flowchart of a microservice-based AI task dynamic deployment and management system proposed in this invention.

[0021] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. Detailed Implementation

[0022] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0023] Example 1: Please see Figure 1As shown, this embodiment is a microservice-based AI task dynamic deployment and management system, including a task management center platform and multiple AI execution nodes communicating with it via a network. The task management center platform is used to uniformly plan and manage the AI ​​task execution topology in the entire computing resource pool, and issue AI task deployment instructions based on the existing task load and resource utilization. Before AI task deployment, the AI ​​execution nodes are tested for runtime environment and model dependencies to ensure deployment compatibility. After deployment, the task runtime status, model version information is collected and stored, and the current execution topology is updated. The task microservices are managed for version control, canary release, and dynamic scaling scheduling. The task management center platform has an AI task scheduling module. The AI ​​task scheduling module is connected to the environment dependency detection module, microservice version management module, execution topology management module, and first communication module. The AI ​​execution nodes are used to receive and execute deployment commands from the task management center platform, becoming usable AI microservice instances. The AI ​​execution nodes use dependency detection and image verification mechanisms to achieve cross-platform deployment, and use service health checks and inference correctness detection mechanisms to verify deployment success. The AI ​​execution nodes include a dependency detection execution module, a microservice deployment execution module, and a second communication module.

[0024] Example 2: Please see Figure 1 As shown, this embodiment is an operation method for a microservice-based AI task dynamic deployment and management system, including the following steps: Step 1: The task management center platform analyzes the current resource status based on the execution topology management module, determines the nodes that need to deploy AI model tasks, and sends the selected node information to the environment dependency detection module; Step 2: The environment dependency detection module communicates with the AI ​​execution node through the first communication module to detect the running environment of the AI ​​execution node, and the execution node returns the detection results; based on the detection results, the deployment phase begins. Step 3: The AI ​​task scheduling module connects to qualified AI execution nodes. The AI ​​execution nodes automatically install the model and inference service images through the microservice deployment and execution module, and return to the task management center after the deployment is completed; the task management center records the deployment results. Step 4: The microservice version management module sends a notification and automatically performs dependency upgrades and model hot updates after confirmation.

[0025] The execution topology management module obtains the current node topology, load, and model distribution, selects the target node based on task requirements, and sends the node credentials to the first communication module. The task management center concurrently connects to multiple AI execution nodes through the first communication module, using a threaded asynchronous communication mode to achieve concurrent deployment of multiple nodes. The dependency detection execution module automatically detects the AI ​​execution node environment and returns the detection results to the task management center. If the node environment does not meet the dependency requirements, it distributes the missing component package and performs automatic installation. The AI ​​task scheduling module distributes the model version and inference image version to the target node, and the AI ​​execution node automatically completes the deployment and returns the results to the task management center.

[0026] The concurrent deployment of the microservice deployment and execution module adopts an event-driven mechanism.

[0027] The dependency detection execution module includes an environment description document and a dependency detection script; the execution node executes the script to automatically detect GPU / CPU, runtime environment, model dependencies and network ports, and generates a structured report that is sent back to the task management center platform.

[0028] The concurrent deployment process includes: integrity verification of model files and image packages, automatic deployment of inference services and execution of health checks, verification of the correctness of example inference, and registration of services to the task management center after successful deployment.

[0029] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

[0030] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. A microservice-based AI task dynamic deployment and management system, characterized in that: The system includes a task management center platform and multiple AI execution nodes that communicate with it via a network. The task management center platform is used to uniformly plan and manage the AI ​​task execution topology across the entire computing resource pool, and to issue AI task deployment instructions based on the existing task load and resource utilization. Before deploying AI tasks, it performs runtime environment and model dependency checks on the AI ​​execution nodes to ensure deployment compatibility. After deployment, it collects and stores task running status and model version information and updates the current execution topology. It also performs version management, canary releases, and dynamic scaling scheduling for task microservices. The task management center platform includes an AI task scheduling module. The AI ​​task scheduling module is connected to the environment dependency detection module, microservice version management module, execution topology management module, and first communication module respectively. The AI ​​execution node is used to receive and execute deployment commands from the task management center platform to become a usable AI microservice instance. The AI ​​execution node uses dependency detection mechanism and image verification mechanism to achieve cross-platform deployment, and uses service health check and inference correctness detection mechanism to verify deployment success. The AI ​​execution node includes a dependency detection execution module, a microservice deployment execution module, and a second communication module.

2. The AI ​​task dynamic deployment and management system based on microservices as described in claim 1, characterized in that: The execution topology management module is used to collect the deployment status, resource usage information and node performance indicators of AI microservice instances, and store them in the topology database, presenting a visual deployment topology through a web interface; the database also stores the access credentials and resource limit information of AI execution nodes; The microservice version management module is used to uniformly store and manage AI model versions, inference image versions, and runtime dependency versions, and maintain model and image repositories. Supports adding, deleting, upgrading, rolling back, canary releases, and version consistency verification; maintains a version record for each AI execution node for subsequent upgrades and management; The environment dependency detection module is used to automatically detect the running environment of the execution node before the AI ​​task is deployed. Based on the detection results, it extracts the corresponding dependent components from the microservice version management module and distributes them to the execution node for deployment. If the dependency is satisfied, the result is passed to the AI ​​task scheduling module. The AI ​​task scheduling module is used to select execution nodes and issue deployment instructions; after confirming that the dependency detection is passed, it applies for the corresponding model version and inference image version from the microservice version management module and automatically deploys them to the execution node; and adopts a concurrent deployment method to achieve efficient batch scheduling. The first communication module is used to establish control channels and model transmission channels between the task management center platform and the execution nodes.

3. The AI ​​task dynamic deployment and management system based on microservices as described in claim 2, characterized in that: The dependency detection execution module is used to parse dependency detection instructions from the task management center platform, automatically execute environment detection scripts, output dependency detection results, and automatically install dependency component packages issued by the center platform and return the results. The dependency detection description document uses YAML; The microservice deployment and execution module is used to automatically pull model files, inference service images and runtime dependencies, parse deployment description documents and automatically complete microservice and model loading and startup; After deployment, health checks and inference correctness verification are performed. Verification methods include cold start latency detection, heartbeat packet detection, example inference verification, or port availability verification. The second communication module includes a high-speed model transmission channel and a status feedback channel; the high-speed model transmission channel is based on an encrypted transmission protocol, and the status feedback channel uses a lightweight message queue to achieve real-time status feedback.

4. The AI ​​task dynamic deployment and management system based on microservices as described in claim 3, characterized in that: The task management center platform runs on a physical server; the AI ​​execution nodes include container instances, virtual machines, physical machines, and edge computing devices.

5. An operational method for a microservice-based AI task dynamic deployment and management system, characterized in that; Operating using the microservice-based AI task dynamic deployment and management system described in claim 4 mainly includes the following steps: Step 1: The task management center platform analyzes the current resource status based on the execution topology management module, determines the nodes that need to deploy AI model tasks, and sends the selected node information to the environment dependency detection module; Step 2: The environment dependency detection module communicates with the AI ​​execution node through the first communication module to detect the running environment of the AI ​​execution node, and the execution node returns the detection results; based on the detection results, the deployment phase begins; Step 3: The AI ​​task scheduling module connects to qualified AI execution nodes. The AI ​​execution nodes automatically install the model and inference service images through the microservice deployment and execution module, and return to the task management center after the deployment is completed; the task management center records the deployment results. Step 4: The microservice version management module sends a notification and automatically performs dependency upgrades and model hot updates after confirmation.

6. The operation method of the AI ​​task dynamic deployment and management system based on microservices according to claim 5, characterized in that: The execution topology management module obtains the current node topology, load, and model distribution, selects the target node based on task requirements, and sends the node credentials to the first communication module; the task management center connects to multiple AI execution nodes concurrently through the first communication module, and uses a threaded asynchronous communication mode to achieve concurrent deployment of multiple nodes; The dependency detection and execution module automatically detects the AI ​​execution node environment and returns the detection results to the task management center; If the node environment does not meet the dependency requirements, the missing component package is distributed and automatic installation is performed; the AI ​​task scheduling module distributes the model version and inference image version to the target node, and the AI ​​execution node automatically completes the deployment and returns the results to the task management center.

7. The AI ​​task dynamic deployment and management system based on microservices as described in claim 6, characterized in that: The concurrent deployment method of the microservice deployment and execution module adopts an event-driven mechanism.

8. The AI ​​task dynamic deployment and management system based on microservices according to claim 7, characterized in that: The dependency detection execution module detects the environment description document and the dependency detection script; the execution node executes the script to automatically detect GPU / CPU, runtime environment, model dependencies and network ports, and generates a structured report to send back to the task management center platform.

9. The AI ​​task dynamic deployment and management system based on microservices as described in claim 8, characterized in that: The concurrent deployment process includes: integrity verification of model files and image packages, automatic deployment of inference services and execution of health checks, verification of the correctness of example inference, and registration of services to the task management center after successful deployment.

Images