An AI model deployment system based on an AI agent

Abstract

The application relates to the field of cloud native and artificial intelligence, and provides an AI model deployment system based on an AI agent, which comprises a natural language interaction system of the AI agent, a Kubernetes-oriented local executor system, an AI model deployment system, a simple test scene deployment system and a standard production scene pipeline deployment system. The application connects the Kubernetes by taking the AI agent as a top layer control, realizes unified control of deployment and management of the AI model in a highly abstract natural language, translates an abstract business intention of a user into a configuration parameter of a bottom layer deployment mode, and reduces an operation threshold of the user. Unified control is provided for two deployment processes, namely simple deployment and pipeline standardization deployment, based on the AI agent.

Description

Technical Field

[0001] This invention relates to the fields of cloud-native and artificial intelligence technologies, and specifically to an AI model deployment system based on AI agents. Background Technology

[0002] In recent years, with the continuous iteration and maturation of artificial intelligence (AI) technology, AI has become the cornerstone of innovation at the forefront of rapidly developing science and technology. As AI technology matures and more and more AI applications are implemented, countries around the world have begun to pay attention to AI applications and have introduced policies to promote AI development. Artificial intelligence is rapidly penetrating into the daily lives of ordinary people, and AI technology is reshaping all walks of life and changing our daily routines.

[0003] The increasing prevalence of AI services across various industries highlights the urgent need for efficient and scalable deployment strategies. Currently, AI services are often delivered to the public through deployment on public clouds. The rapid growth and development of cloud-based AI is primarily due to the increasing adoption of generative artificial intelligence and machine learning across industries. By integrating AI with cloud computing, cloud-based AI deployment allows enterprises and individual developers to fully leverage the readily available infrastructure and resources to develop their AI products.

[0004] In the cloud computing field, Kubernetes, as the benchmark in container orchestration, has become the foundation of modern cloud-native infrastructure. It is the cornerstone of all cloud services, and according to data from the CNCF Cloud Native Foundation's 2024 annual report, its market share exceeded an astonishing 80%. While it is an essential component of cloud-native computing, its management and configuration are complex. It uses complex declarative YAML files for resource configuration, requiring users deploying AI models to possess sufficient knowledge of Kubernetes command-line tools and YAML configuration files. This makes it extremely difficult for ordinary developers and users to actually deploy AI services, regardless of the size and complexity of the AI ​​model itself. Furthermore, Kubernetes is essentially a general-purpose infrastructure, not fully adapted and customized for AI model inference scenarios. Its elastic scaling metrics are singular, it lacks a standard inference protocol, and its model configuration for different training architectures is complex. In summary, Kubernetes has significant inconveniences and shortcomings in resource operation and configuration, the differentiated adaptation of different AI model dependency frameworks (such as TensorFlow and PyTorch), the standardized external opening of APIs after successful model deployment, AI service resource scheduling and monitoring under different traffic scenarios, the management of AI model-related files, and image construction.

[0005] To address the aforementioned issues related to AI model deployment, existing technologies have optimized three aspects: AI model deployment process and methods, AI model management system, and AI model resource scheduling and data collection.

[0006] The deployment process and methods of AI models mainly refer to the deployment process and methods used before an AI model is successfully deployed and provides AI services to the outside world.

[0007] In response to the differences in the dependent frameworks of different types of AI models, specific format models exported from different training frameworks (such as TensorFlow, PyTorch, and MindSpore) can be extracted, their operators can be mapped and transformed into a unified intermediate representation layer (IR), and computation graph optimization can be performed at the IR layer, such as operator fusion (merging convolution and activation functions) and dead code elimination. During deployment, the IR can be automatically compiled into executable machine code that is most suitable for the hardware type of the target physical machine, thereby unifying the interface to achieve cross-platform deployment of AI models.

[0008] To address the complexity and inconvenience of Kubernetes resource operations and configurations, an AI-based question-and-answer approach can be used. Deployers input natural language commands through a dialogue interface. The dialogue system's built-in large model parsing command extracts key parameters (such as concurrency requirements, model paths, and base images) and injects these parameters into a pre-defined deployment template (Kubernetes YAML file or HelmChart). The generated configuration file undergoes static syntax checking, and once confirmed to be error-free, the deployment operation is executed by calling the K8s API.

[0009] AI model management refers to the comprehensive management of AI models from deployment, launch, and iteration. It mainly focuses on the centralized and standardized management of AI model weight files, images, service code, etc., thereby reducing the operational complexity for deployers.

[0010] For AI deployment management systems, a WebGUI console is provided. Users fill in model information and upload model weight files on the interface. The management system automatically triggers the CI / CD process in the background, packaging the uploaded model files, the basic inference engine, and service code into a standardized Docker image and pushing it to a private image repository. The system records all version information of the model, allowing administrators to perform lifecycle management operations such as "start, stop, canary upgrade, and rollback" by clicking buttons on the interface. At the underlying level, these are translated into specific container orchestration instructions.

[0011] The above deployment methods and systems address some issues specifically in terms of AI model deployment methods and deployment management, but the following problems and shortcomings still exist: The existing deployment system lacks a higher level of abstraction in its natural language deployment control. Its interaction level is too low; even with AI assistance, its capabilities are limited to code completion, static script generation, and YAML resource file review, presenting an extremely high operational barrier. Users must master complex YAML resource configuration methods, precise Kubernetes command syntax, and so on. Furthermore, specific command syntax must be used when switching between modules or making unified calls, making it impossible to deploy services on Kubernetes through natural language interaction with AI agents. The operational barrier for users is excessively high. There is a lack of deployment path options for rapid verification and standardized pipeline scenarios. The current approach only provides two separate modes for service deployment: simple deployment and pipeline deployment. Simple deployment does not support switching to pipeline deployment, and pipeline deployment mostly relies on standardized pipelines and predefined configurations. While suitable for standardized delivery, it lacks sufficient support for ad-hoc verification, lightweight testing, or rapid trial deployment scenarios, making it inconvenient for users to perform simple test deployments. Summary of the Invention

[0012] To address the aforementioned technical problems, this invention aims to provide an AI model deployment system based on AI agents. This invention employs the following technical solutions: An AI model deployment system based on AI agents includes a natural language interaction system for AI agents, a local executor system for Kubernetes, an AI model deployment system, a simple test scenario deployment system, and a standard production scenario pipeline deployment system.

[0013] Optionally, a natural language interaction system may include an interaction and entry layer, a context and memory layer, a core orchestration layer, a protocol adaptation and conversion layer, and a tool execution engine layer.

[0014] Optionally, the interaction and entry layer is responsible for receiving user natural language commands, uploaded files and deployment parameters, and organizing them into a unified task object; for user-submitted YAML files, this layer identifies apiVersion, kind, metadata.name, image field, resource field and storage field, forms a summary and sends it to the subsequent processing stage; The context and memory layer uses an embedded SQLite database to store session metadata, user input summaries, confirmed deployment parameters, uploaded file summaries, and tool execution trajectories, enabling the AI ​​agent to maintain its task state during multi-round interactions. The core orchestration layer maintains the lifecycle of deployment tasks using a state machine, determines whether the current task belongs to a simple test deployment path or a standard pipeline deployment path, and decides on the set of tools to be invoked. The protocol adaptation and conversion layer converts the internal task structure, context summary, and tool definition into a function call structure that the large language model can recognize, and converts the model's return results into internally executable tool call requests; The tool execution engine layer is responsible for executing local controlled commands and external MCP tool calls, and for sending the execution status, error messages and structured results back to the core orchestration layer.

[0015] Optionally, the Kubernetes-based local executor system sets up the local executor in a host environment where the Kubectl command-line tool is deployed; Kubectl is a command-line client for Kubernetes, used to send resource query, creation, update, and deletion requests to the Kubernetes API Server; The local executor encapsulates executable capabilities into a fixed tool interface.

[0016] Optionally, the steps of encapsulating executable capabilities into a fixed tool interface include: k8s_list_resources resource list query tool, k8s_describe_resource resource details query tool, k8s_get_logs log reading tool, k8s_get_events event query tool, k8s_save_manifest manifest saving tool, k8s_apply_manifest manifest application tool, k8s_delete_resource resource deletion tool; Optionally, a native executor system for Kubernetes includes a tool interface layer, a command construction layer, a process execution layer, and a result normalization layer.

[0017] The tool interface layer uses JSON Schema to describe and validate the tool name, function description, parameter mode, calling method, and return format. The command construction layer converts the structured parameters passed from the upper layer into an array of Kubectl parameters, converts resource query requests into an array of parameters containing kubectl, get, resource type, namespace, and output format, and converts resource deployment requests into an array of parameters containing kubectl, apply, -f, and file path. This layer treats fields such as resource type, name, namespace, and output format as plain data, preventing the AI ​​agent from directly outputting executable scripts. The process execution layer loads the Kubeconfig cluster access configuration file and target namespace, starts a controlled child process in the host environment to execute Kubectl, and is responsible for writing YAML temporary files, variable substitution and a small amount of text processing. For apply, delete and patch write operations, the core orchestration layer needs to provide permission confirmation flags, and deletion operations require secondary confirmation from the user. The standardization layer organizes the original stdout standard output, stderr standard error, return code, and timeout into a unified result, and maps resource non-existence, insufficient permissions, YAML format error, image pull failure, scheduling failure, probe anomaly, and timeout not ready to the standard anomaly type, which facilitates the AI ​​agent to continue diagnosis.

[0018] Optionally, the deployment process for the AI ​​model deployment system includes: Users input deployment requirements into the AI ​​agent and save the deployment configuration. The AI ​​agent selects a simple deployment path or a standardized deployment path based on the task type. A simple deployment path involves direct interaction with the K8S cluster via Kubectl and Linux I / O. The standardized deployment path uses the MCP Server to call pipeline components to complete artifact building and configuration synchronization; Once deployed, model service requests enter the K8S cluster. Prometheus monitors the resource status and provides resource monitoring results to the AI ​​agent. The system is designed around AI models and integrates with GitLab, GitLab Registry, MinIO, ArgoCD, Prometheus, PostgreSQL, and Kubernetes components. GitLab is used to store inference service code, container build files, Kubernetes manifest templates, GitOps application definitions, and CI / CD continuous integration / continuous delivery pipeline configurations. GitLab Registry serves as a private image repository for storing images that have been built from the pipeline. MinIO is used as an object storage to save large model weight files. ArgoCD performs continuous GitOps synchronization within a Kubernetes cluster, synchronizing the declarative manifest in the configuration repository to the target cluster. Prometheus collects monitoring metrics such as cluster nodes, workload, request volume, and latency. PostgreSQL stores model deployment registration information and runtime parameters; Kubectl interacts with the Kubernetes API Server through a local executor; GitLab, MinIO, ArgoCD, Prometheus, and PostgreSQL are suitable for accessing AI agents through an MCP server; AI agents can read or generate repository files, create branches, commit changes, trigger pipelines, and reclaim pipeline states through GitLab MCP. Use ArgoCD MCP to query application status, trigger synchronization, and read difference status. The system uses PostgreSQL MCP to write model registration information, image summary, model file path, service object identifier, and deployment status. Read pre-deployment capacity information and post-deployment acceptance metrics using Prometheus MCP; Kubectl provides local execution capabilities through built-in tools for direct deployment, status query, log diagnostics, and deployment acceptance. The system sets up the `model_deployment_registry` table in PostgreSQL, which stores the following fields: `model_name` (model name), `namespace` (namespace), `model_type` (model type), `deployment_mode` (deployment mode), `gitlab_project_path` (project path), `git_ref` (branch), `manifest_path` (manifest path), `image_ref` (image address), `image_digest` (image digest), `minio_bucket` (model bucket name), `minio_object_key` (model file path), `service_name` (service name), `configmap_name` (configuration object name), and `deploy_status` (deployment status), ensuring that model deployment results are queryable and traceable.

[0019] Optionally, the deployment sequence for simple test scenarios of the system includes: After the user submits the model deployment YAML template, service exposure template, Secret key object or ConfigMap configuration object template, target namespace, image address, model file path and resource parameters, the interaction and entry layer extracts the key information of InferenceService inference service object, Deployment workload object, Service service object and Secret key object from the template, and parses the target name, namespace, image field and resource field; Subsequently, the context and memory layer writes the template summary, user-added parameters, and target environment into the SQLite session storage. The core orchestration layer determines that the task is a test direct deployment task and selects the local executor toolchain. The AI ​​agent generates the final YAML manifest text based on the current parameters and calls the k8s_save_manifest tool to securely write the manifest to the isolated working directory. After the local executor returns the file path, the core orchestration layer calls the k8s_apply_manifest tool, which loads Kubeconfig and distributes resources via Kubectl. After the resource is created, the AI ​​agent checks the object status using k8s_list_resources and k8s_describe_resource; For model services based on KServe, the InferenceService status, service entry point, and key readiness conditions are also checked. If an anomaly occurs during deployment, the AI ​​agent calls k8s_get_events and k8s_get_logs to obtain the event stream and log fragments, and provides a diagnostic conclusion based on the results of previous executions. Subsequently, the context and memory layer writes the template summary, user-added parameters, and target environment to the SQLite session store. The core orchestration layer determines that the task is a test direct deployment task and selects the local executor toolchain. The AI ​​agent generates the final YAML manifest text based on the current parameters and calls the k8s_save_manifest tool to safely write the manifest to the isolated working directory. After the local executor returns the file path, the core orchestration layer calls the k8s_apply_manifest tool, which loads Kubeconfig and distributes resources via Kubectl. After the resource is created, the AI ​​agent checks the object status using k8s_list_resources and k8s_describe_resource; For model services based on KServe, the InferenceService status, service entry point, and key readiness conditions are also checked. If an anomaly occurs during deployment, the AI ​​agent calls k8s_get_events and k8s_get_logs to obtain the event stream and log fragments, and provides a diagnostic conclusion based on the results of previous executions. This path can complete template verification and model service trial deployment within a relatively short link, making it suitable as a verification entry point before standard pipeline deployment.

[0020] Optionally, the standard production scenario pipeline deployment system provides standard pipeline deployment for production delivery scenarios. The standard pipeline deployment sequence includes seven stages: The first phase involves compiling deployment requirements; The second phase involves organizing warehouse templates; The third phase consists of image building, pushing, and inventory updating; The fourth phase is GitOps synchronous deployment; The fifth stage is cluster deployment and resource creation; The sixth stage is deployment and acceptance; The seventh stage involves deployment registration and presetting of operating parameters.

[0021] Optionally, deployment requirements compilation includes: extracting the model name, model type, inference framework, image base, model file source, target environment, whether to use Knative serverless mode, and resource constraint parameters from the AI ​​agent, and writing them into the session context; The repository template organization includes: the AI ​​agent reads the template repository through GitLab MCP and generates or completes the inference service code, Dockerfile container build script, KServe or Knative resource template, ArgoCDApplication application manifest and GitLab CI / CD pipeline configuration in the target project; Image building, pushing, and manifest updates include: GitLab Runner executes a pipeline in an isolated container of Docker Executor, first downloading the model weight file from MinIO, then combining the inference service code, Dockerfile, and model file to build the image, and pushing it to the GitLab Registry; after the pipeline is completed, it automatically updates the image tag in the Kubernetes YAML manifest and commits it to the configuration repository; GitOps synchronous deployment includes: GitLab notifies ArgoCD via Webhook, ArgoCD pulls the latest declarative manifest, compares the target state with the current cluster state, and synchronizes the changes to the Kubernetes cluster; Cluster deployment and resource creation include: Kubernetes nodes pull private images through pre-configured imagePullSecrets, and create workloads, service objects, and inference endpoints; Deployment acceptance includes: the AI ​​agent, combined with the local executor, Prometheus, and log query capabilities, checks the status of resource objects, Pod readiness status, endpoint health status, and minimal inference call results; For model services that comply with Open Inference Protocol V2, the system performs health checks, metadata reading, sample calls, and return result verification; for generative models, the system performs lightweight text generation verification through an OpenAI-style interface. Deployment registration and pre-setting of running parameters include: the AI ​​agent writes the model source, image address, image summary, MinIO object path, service object identifier and deployment status into PostgreSQL, and creates or updates the corresponding ConfigMap running configuration object, providing an entry point for monitoring, parameter tuning and elastic policy takeover after the model goes online.

[0022] The present invention has the following beneficial effects: By using an AI agent as the top-level control and connecting to Kubernetes, it achieves unified control over the deployment and management of AI models through highly abstract natural language. This translates the user's abstract business intent into configuration parameters for the underlying deployment mode, lowering the operational threshold for users. Based on the AI ​​agent, it provides unified control over two deployment processes tailored to user needs: simple deployment and pipelined standardized deployment. Attached Figure Description The present invention will be further described with reference to the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present invention. For those skilled in the art, other drawings can be obtained based on the following drawings without creative effort.

[0023] Figure 1 This is an architecture diagram of an AI agent in an AI model deployment system based on AI agents according to the present invention; Figure 2 This is a flowchart of the natural language interaction process in this invention; Figure 3 This is a flowchart of the local executor invocation process in this invention; Figure 4 This is a diagram showing the relationship between system components in this invention; Figure 5 This is a flowchart of the AI ​​model deployment process in this invention; Figure 6 This is a sequence diagram of the deployment of a simple test scenario in this invention; Figure 7 This is a timing diagram of the standard pipeline deployment in this invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] In the description of this invention, it should be noted that the terms "vertical," "upper," "lower," "horizontal," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0026] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or a connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0027] Artificial intelligence (AI) is being widely adopted across various industries. As these models move from theoretical concepts to practical applications, the ability to effectively deploy and manage them becomes crucial. Traditional deployment methods often face significant challenges in adapting to evolving needs and ensuring cost-effectiveness. While existing technologies have addressed issues such as the complexity and low adaptability of AI model deployment to some extent, there remains considerable room for optimization and iteration as technology advances.

[0028] This invention addresses the shortcomings of existing technologies in both the deployment process and management of AI models by proposing an AI model deployment system based on AI agents. It introduces AI agents as the top-level control layer, enabling direct deployment of AI models on Kubernetes using natural language. Furthermore, it categorizes and optimizes AI models based on their characteristics, providing users with a highly efficient, highly abstract, and low-barrier method for deploying AI models.

[0029] The AI ​​model deployment system proposed in this invention uses AI agents as the unified control entry point for deployment tasks.

[0030] AI stands for Artificial Intelligence; K8S is the Kubernetes container orchestration system; API stands for Application Programming Interface; YAML is a configuration file-oriented data description language; MCP stands for Model Context Protocol.

[0031] In the AI ​​model deployment system, users can submit information such as model name, target namespace, mirror address, model file URI (Uniform Resource Identifier), resource constraints, deployment environment, and whether a standard pipeline is used via natural language. Alternatively, they can directly submit existing YAML templates.

[0032] like Figures 1-7 As shown, an AI model deployment system based on AI agents includes a natural language interaction system for AI agents, a local executor system for Kubernetes, an AI model deployment system, a simple test scenario deployment system, and a standard production scenario pipeline deployment system.

[0033] Optionally, a natural language interaction system may include an interaction and entry layer, a context and memory layer, a core orchestration layer, a protocol adaptation and conversion layer, and a tool execution engine layer.

[0034] Optionally, the interaction and entry layer is responsible for receiving user natural language commands, uploaded files and deployment parameters, and organizing them into a unified task object; for user-submitted YAML files, this layer identifies apiVersion, kind, metadata.name, image field, resource field and storage field, forms a summary and sends it to the subsequent processing stage; The context and memory layer uses an embedded SQLite database to store session metadata, user input summaries, confirmed deployment parameters, uploaded file summaries, and tool execution trajectories, enabling the AI ​​agent to maintain its task state during multi-round interactions. The core orchestration layer maintains the lifecycle of deployment tasks using a state machine, determines whether the current task belongs to a simple test deployment path or a standard pipeline deployment path, and decides on the set of tools to be invoked. The protocol adaptation and conversion layer converts the internal task structure, context summary, and tool definition into a function call structure that the large language model can recognize, and converts the model's return results into internally executable tool call requests; The tool execution engine layer is responsible for executing local controlled commands and external MCP tool calls, and for sending the execution status, error messages and structured results back to the core orchestration layer.

[0035] Through a natural language interaction system, users' business intentions can be gradually translated into Kubernetes resource operations, repository operations, image building operations, and deployment acceptance operations.

[0036] To enable AI agents to operate Kubernetes clusters in a controlled manner, an optional Kubernetes-based local executor system can be set up in a host environment where the Kubectl command-line tool is deployed. Kubectl is a command-line client for Kubernetes, used to send resource query, creation, update, and deletion requests to the Kubernetes API Server; The local executor does not allow AI agents to directly concatenate and execute arbitrary terminal commands; rather, it encapsulates executable capabilities into a fixed tool interface.

[0037] Optionally, the steps of encapsulating executable capabilities into a fixed tool interface include: k8s_list_resources resource list query tool, k8s_describe_resource resource details query tool, k8s_get_logs log reading tool, k8s_get_events event query tool, k8s_save_manifest manifest saving tool, k8s_apply_manifest manifest application tool, k8s_delete_resource resource deletion tool; Optionally, a native executor system for Kubernetes includes a tool interface layer, a command construction layer, a process execution layer, and a result normalization layer.

[0038] Optionally, the tool interface layer uses JSON Schema (i.e., JSON structure) to describe and validate the tool name, function description, parameter mode, calling method, and return format. The command construction layer converts the structured parameters passed from the upper layer into an array of Kubectl parameters, converts resource query requests into an array of parameters containing kubectl, get, resource type, namespace, and output format, and converts resource deployment requests into an array of parameters containing kubectl, apply, -f, and file path. This layer treats fields such as resource type, name, namespace, and output format as plain data, preventing the AI ​​agent from directly outputting executable scripts. The process execution layer loads the Kubeconfig cluster access configuration file and target namespace, starts a controlled child process in the host environment to execute Kubectl, and is responsible for writing YAML temporary files, variable substitution and a small amount of text processing. For apply, delete and patch write operations, the core orchestration layer needs to provide permission confirmation flags, and deletion operations require secondary confirmation from the user. The standardization layer organizes the original stdout standard output, stderr standard error, return code, and timeout into a unified result, and maps resource non-existence, insufficient permissions, YAML format error, image pull failure, scheduling failure, probe anomaly, and timeout not ready to the standard anomaly type, which facilitates the AI ​​agent to continue diagnosis.

[0039] Optionally, the deployment process for the AI ​​model deployment system includes: Users input deployment requirements into the AI ​​agent and save the deployment configuration. The AI ​​agent selects a simple deployment path or a standardized deployment path based on the task type. A simple deployment path involves direct interaction with the K8S cluster via Kubectl and Linux I / O. The standardized deployment path uses the MCP Server to call pipeline components to complete artifact building and configuration synchronization; Once deployed, model service requests enter the K8S cluster. Prometheus monitors the resource status and provides resource monitoring results to the AI ​​agent. The system is designed around AI models and integrates with GitLab, GitLab Registry, MinIO, ArgoCD, Prometheus, PostgreSQL, and Kubernetes components. GitLab is used to store inference service code, container build files, Kubernetes manifest templates, GitOps (a continuous delivery approach based on declarative configuration in a Git repository) application definitions, and CI / CD continuous integration / continuous delivery pipeline configurations. GitLab Registry serves as a private image repository for storing images that have been built from the pipeline. MinIO is used as an object storage to save large model weight files. ArgoCD performs continuous GitOps synchronization within a Kubernetes cluster, synchronizing the declarative manifest in the configuration repository to the target cluster. Prometheus collects monitoring metrics such as cluster nodes, workload, request volume, and latency. PostgreSQL stores model deployment registration information and runtime parameters; Kubectl interacts with the Kubernetes API Server through a local executor; GitLab, MinIO, ArgoCD, Prometheus, and PostgreSQL are suitable for accessing AI agents via an MCP server; AI agents can read or generate repository files, create branches, commit changes, trigger pipelines, and reclaim pipeline states through GitLab MCP. Use ArgoCD MCP to query application status, trigger synchronization, and read difference status. The system uses PostgreSQL MCP to write model registration information, image summary, model file path, service object identifier, and deployment status. Read pre-deployment capacity information and post-deployment acceptance metrics using Prometheus MCP; Kubectl provides local execution capabilities through built-in tools for direct deployment, status query, log diagnostics, and deployment acceptance. The system sets up the `model_deployment_registry` table in PostgreSQL, which stores the following fields: `model_name` (model name), `namespace` (namespace), `model_type` (model type), `deployment_mode` (deployment mode), `gitlab_project_path` (project path), `git_ref` (branch), `manifest_path` (manifest path), `image_ref` (image address), `image_digest` (image digest), `minio_bucket` (model bucket name), `minio_object_key` (model file path), `service_name` (service name), `configmap_name` (configuration object name), and `deploy_status` (deployment status), ensuring that model deployment results are queryable and traceable.

[0040] The Simple Test Scenario Deployment System provides a lightweight, direct deployment path for functional verification, API integration testing, sample demonstrations, and rapid template trial-and-error scenarios. This path requires that the images and model files are already accessible to the cluster, focusing on quickly completing Kubernetes resource orchestration without entering image building, GitOps synchronization, and version auditing processes. Optionally, the deployment sequence for the Simple Test Scenario Deployment System includes: After the user submits the model deployment YAML template, service exposure template, Secret key object or ConfigMap configuration object template, target namespace, image address, model file path and resource parameters, the interaction and entry layer extracts the key information of InferenceService inference service object, Deployment workload object, Service service object and Secret key object from the template, and parses the target name, namespace, image field and resource field; Subsequently, the context and memory layer writes the template summary, user-added parameters, and target environment into the SQLite session storage. The core orchestration layer determines that the task is a test direct deployment task and selects the local executor toolchain. The AI ​​agent generates the final YAML manifest text based on the current parameters and calls the k8s_save_manifest tool to securely write the manifest to the isolated working directory. After the local executor returns the file path, the core orchestration layer calls the k8s_apply_manifest tool, which loads Kubeconfig and distributes resources via Kubectl. After the resource is created, the AI ​​agent checks the object status using k8s_list_resources and k8s_describe_resource; For model services based on KServe, the InferenceService status, service entry point, and key readiness conditions are also checked. If an anomaly occurs during deployment, the AI ​​agent calls k8s_get_events and k8s_get_logs to obtain the event stream and log fragments, and provides a diagnostic conclusion based on the results of previous executions. Subsequently, the context and memory layer writes the template summary, user-added parameters, and target environment to the SQLite session store. The core orchestration layer determines that the task is a test direct deployment task and selects the local executor toolchain. The AI ​​agent generates the final YAML manifest text based on the current parameters and calls the k8s_save_manifest tool to safely write the manifest to the isolated working directory. After the local executor returns the file path, the core orchestration layer calls the k8s_apply_manifest tool, which loads Kubeconfig and distributes resources via Kubectl. After the resource is created, the AI ​​agent checks the object status using k8s_list_resources and k8s_describe_resource; For model services based on KServe, the InferenceService status, service entry point, and key readiness conditions are also checked. If an anomaly occurs during deployment, the AI ​​agent calls k8s_get_events and k8s_get_logs to obtain the event stream and log fragments, and provides a diagnostic conclusion based on the results of previous executions. This path can complete template verification and model service trial deployment within a relatively short link, making it suitable as a verification entry point before standard pipeline deployment.

[0041] Optionally, the standard production scenario pipeline deployment system provides standard pipeline deployment for production delivery scenarios. The standard pipeline deployment sequence includes seven stages: The first phase involves compiling deployment requirements; The second stage is the organization of warehouse templates; The third phase consists of image building, pushing, and inventory updating; The fourth phase is GitOps synchronous deployment; The fifth stage is cluster deployment and resource creation; The sixth stage is deployment and acceptance; The seventh stage involves deployment registration and pre-setting of operating parameters.

[0042] Optionally, deployment requirements compilation includes: extracting the model name, model type, inference framework, image base, model file source, target environment, whether to use Knative serverless mode, and resource constraint parameters from the AI ​​agent, and writing them into the session context; The repository template organization includes: the AI ​​agent reads the template repository through GitLab MCP and generates or completes the inference service code, Dockerfile container build script, KServe or Knative resource template, ArgoCDApplication application manifest and GitLab CI / CD pipeline configuration in the target project; Image building, pushing, and manifest updates include: GitLab Runner executes a pipeline in an isolated container of Docker Executor, first downloading the model weight file from MinIO, then combining the inference service code, Dockerfile, and model file to build the image, and pushing it to the GitLab Registry; after the pipeline is completed, it automatically updates the image tag in the Kubernetes YAML manifest and commits it to the configuration repository; GitOps synchronous deployment includes: GitLab notifies ArgoCD via Webhook, ArgoCD pulls the latest declarative manifest, compares the target state with the current cluster state, and synchronizes the changes to the Kubernetes cluster; Cluster deployment and resource creation include: Kubernetes nodes pull private images through pre-configured imagePullSecrets, and create workloads, service objects, and inference endpoints; Deployment acceptance includes: the AI ​​agent, combined with the local executor, Prometheus, and log query capabilities, checks the status of resource objects, Pod readiness status, endpoint health status, and minimal inference call results; For model services that comply with Open Inference Protocol V2, the system performs health checks, metadata reading, sample calls, and return result verification; for generative models, the system performs lightweight text generation verification through an OpenAI-style interface. Deployment registration and pre-setting of running parameters include: the AI ​​agent writes the model source, image address, image summary, MinIO object path, service object identifier and deployment status into PostgreSQL, and creates or updates the corresponding ConfigMap running configuration object, providing an entry point for monitoring, parameter tuning and elastic policy takeover after the model goes online.

[0043] Key points: The AI ​​agent serves as the top-level control entry point for AI model deployment. The system forms a closed-loop execution chain through an interaction and entry layer, a context and memory layer, a core orchestration layer, a protocol adaptation and conversion layer, and a tool execution engine layer, unifying natural language deployment requirements, YAML templates, historical context, and tool call results into task status management. In existing technologies, even with AI assistance, the focus is mostly on question-and-answer interpretation, script generation, or static configuration generation, requiring manual coordination between Kubernetes, image repositories, code repositories, and monitoring systems. This invention, through its core orchestration layer, determines the deployment path, selects tools, triggers diagnostics, and organizes result output, making the AI ​​agent a unified control layer that runs through deployment request parsing, resource operations, pipeline triggering, and deployment acceptance. This invention provides a controlled local executor for Kubernetes operations. It encapsulates Kubectl and system I / O capabilities into fixed tools, employing a tool registry, JSON Schema parameter constraints, parameter array-based command construction, temporary inventory storage, and a standardized result system. Existing solutions typically require users to directly manipulate Kubectl or for the system to generate complete command text, which easily leads to problems such as non-standard command concatenation, unclear execution boundaries, and difficulty in further reasoning based on error feedback. This invention divides resource querying, log reading, event querying, inventory saving, inventory application, and resource deletion into clearly defined tools, and sets up a confirmation system for write operations and standard exception types for error results. Technically, this ensures that the AI ​​agent can only complete deployment and diagnosis within the scope of the registered tools. This invention unifies the orchestration of simple test deployment paths and standard pipeline deployment paths. Existing solutions typically only provide direct deployment for rapid trials or fixed pipelines for production delivery, lacking automatic selection and state transitions between the two types of paths under a common entry point. This invention uses an AI agent to select either a lightweight direct deployment path or a standard production pipeline deployment path based on user intent, target environment, and parameter completeness. The simple test path directly completes YAML manifest writing, kubectl apply distribution, and event log diagnosis through a local executor. The standard production path completes model file acquisition, image building, image pushing, GitOps synchronization, deployment acceptance, and registration through GitLab, MinIO, GitLab Registry, ArgoCD, Kubernetes, and PostgreSQL. Both paths share a natural language entry point, context storage, tool registration, and result diagnosis system, supporting both rapid verification and auditable, standardized delivery.

[0044] Beneficial effects: This invention integrates an AI agent, a local executor, and an MCP component into the system, translating user natural language deployment intentions into controlled Kubernetes resource operations, repository operations, image building operations, GitOps synchronization operations, and deployment acceptance operations. Compared to deployment methods that require users to directly write complex YAML manifests, memorize Kubectl commands, and manually switch between multiple platforms, this invention sets up an AI agent as a unified control entry point at the interaction layer and a fixed tool interface and parameterized command construction system at the execution layer. This enables deployment tasks to be completed through natural language interaction, while avoiding execution boundary issues caused by the AI ​​agent directly concatenating arbitrary commands. Users only need to provide business parameters such as model name, image address, model file path, resource constraints, and target environment, and the system can complete parameter completion, path selection, configuration generation, resource distribution, status query, and anomaly diagnosis, thereby reducing the dependence of AI model deployment on Kubernetes knowledge and pipeline operation and maintenance experience. This invention, through the coexistence of a simple test deployment path and a standard production pipeline deployment path, enables the same system to adapt to both rapid verification and standardized delivery requirements. For test scenarios, the system uses `k8s_save_manifest` and `k8s_apply_manifest` to complete manifest storage and direct deployment, and quickly provides deployment status feedback through resource queries, event queries, and log reading, facilitating model example demonstrations, template debugging, and interface integration. For production scenarios, the system forms a closed-loop pipeline using GitLab, GitLab CI / CD, GitLab Registry, MinIO, ArgoCD, Kubernetes, and PostgreSQL, incorporating code repositories, model file repositories, image repositories, configuration repositories, cluster synchronization, and deployment registration into a unified orchestration process. This ensures that the deployment process includes version auditing, difference comparison, image tracking, state persistence, and a foundation for subsequent rollback. This invention extends the determination of successful AI model deployment from resource object creation to service availability acceptance. After deployment, the system combines Kubernetes status, Prometheus monitoring metrics, inference endpoint health checks, model metadata reading, and minimum sample calls for confirmation. It can identify situations such as objects being created but models not fully loaded, image pull failures, inaccessible model weights, insufficient resources, or probe anomalies. Through a result standardization layer and anomaly type mapping, the AI ​​agent can prioritize calling event and log tools for further diagnosis, generating easily understandable explanations and handling suggestions. Thus, this invention not only automates the deployment of model services but also improves the accuracy and interpretability of deployment result feedback.

[0045] The components, modules, mechanisms, and devices in this invention that are not described in detail are all general standard parts or components known to those skilled in the art. Their structures and principles can be learned by those skilled in the art through technical manuals or conventional experimental methods.

[0046] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the essence and scope of the technical solutions of the present invention.

Claims

1. An AI model deployment system based on AI intelligent agents, characterized in that, This includes a natural language interaction system for AI agents, a local executor system for Kubernetes, an AI model deployment system, a simple test scenario deployment system, and a standard production scenario pipeline deployment system.

2. The AI ​​model deployment system based on AI intelligent agents according to claim 1, characterized in that, A natural language interaction system includes an interaction and entry layer, a context and memory layer, a core orchestration layer, a protocol adaptation and conversion layer, and a tool execution engine layer.

3. The AI ​​model deployment system based on AI intelligent agents according to claim 2, characterized in that, The interaction and entry layer is responsible for receiving user natural language commands, uploaded files, and deployment parameters, and organizing them into a unified task object; for user-submitted YAML files, this layer identifies apiVersion, kind, metadata.name, image field, resource field, and storage field, forms a summary, and sends it to the subsequent processing stage; The context and memory layer uses an embedded SQLite database to store session metadata, user input summaries, confirmed deployment parameters, uploaded file summaries, and tool execution trajectories, enabling the AI ​​agent to maintain its task state during multi-round interactions. The core orchestration layer maintains the lifecycle of deployment tasks using a state machine, determines whether the current task belongs to a simple test deployment path or a standard pipeline deployment path, and decides on the set of tools to be invoked. The protocol adaptation and conversion layer converts the internal task structure, context summary, and tool definition into a function call structure that the large language model can recognize, and converts the model's return results into internally executable tool call requests; The tool execution engine layer is responsible for executing local controlled commands and external MCP tool calls, and for sending the execution status, error messages and structured results back to the core orchestration layer.

4. The AI ​​model deployment system based on AI intelligent agents according to claim 3, characterized in that, The local executor system for Kubernetes sets up the local executor in a host environment where the Kubectl command-line tool is deployed; Kubectl is a command-line client for Kubernetes, used to send resource query, creation, update, and deletion requests to the Kubernetes API Server; The local executor encapsulates executable capabilities into a fixed tool interface.

5. The AI ​​model deployment system based on AI agents according to claim 4, characterized in that, The steps involved in encapsulating executable capabilities into a fixed tool interface include: k8s_list_resources resource list query tool, k8s_describe_resource resource details query tool, k8s_get_logs log reading tool, k8s_get_events event query tool, k8s_save_manifest manifest saving tool, k8s_apply_manifest manifest application tool, and k8s_delete_resource resource deletion tool.

6. The AI ​​model deployment system based on AI intelligent agents according to claim 5, characterized in that, The native executor system for Kubernetes consists of a tool interface layer, a command construction layer, a process execution layer, and a result normalization layer. The tool interface layer uses JSON Schema to describe and validate the tool name, function description, parameter mode, calling method, and return format. The command construction layer converts the structured parameters passed from the upper layer into an array of Kubectl parameters, converts resource query requests into an array of parameters containing kubectl, get, resource type, namespace, and output format, and converts resource deployment requests into an array of parameters containing kubectl, apply, -f, and file path. This layer treats fields such as resource type, name, namespace, and output format as plain data, preventing the AI ​​agent from directly outputting executable scripts. The process execution layer loads the Kubeconfig cluster access configuration file and target namespace, starts a controlled child process in the host environment to execute Kubectl, and is responsible for writing YAML temporary files, variable substitution and a small amount of text processing. For apply, delete and patch write operations, the core orchestration layer needs to provide permission confirmation flags, and deletion operations require secondary confirmation from the user. The standardization layer organizes the original stdout standard output, stderr standard error, return code, and timeout into a unified result, and maps resource non-existence, insufficient permissions, YAML format error, image pull failure, scheduling failure, probe anomaly, and timeout not ready to the standard anomaly type, which facilitates the AI ​​agent to continue diagnosis.

7. The AI ​​model deployment system based on AI intelligent agents according to claim 6, characterized in that, The deployment process of an AI model deployment system includes: Users input deployment requirements into the AI ​​agent and save the deployment configuration. The AI ​​agent selects a simple deployment path or a standardized deployment path based on the task type. A simple deployment path involves direct interaction with the K8S cluster via Kubectl and Linux I / O. The standardized deployment path uses the MCP Server to call pipeline components to complete artifact building and configuration synchronization; Once deployed, model service requests enter the K8S cluster. Prometheus monitors the resource status and provides resource monitoring results to the AI ​​agent. The system is designed around AI models and integrates with GitLab, GitLab Registry, MinIO, ArgoCD, Prometheus, PostgreSQL, and Kubernetes components. GitLab is used to store inference service code, container build files, Kubernetes manifest templates, GitOps application definitions, and CI / CD continuous integration / continuous delivery pipeline configurations. GitLab Registry serves as a private image repository for storing images that have been built from the pipeline. MinIO is used as an object storage to save large model weight files. ArgoCD performs continuous GitOps synchronization within a Kubernetes cluster, synchronizing the declarative manifest in the configuration repository to the target cluster. Prometheus collects monitoring metrics such as cluster nodes, workload, request volume, and latency. PostgreSQL stores model deployment registration information and runtime parameters; Kubectl interacts with the Kubernetes API Server through a local executor; GitLab, MinIO, ArgoCD, Prometheus, and PostgreSQL are suitable for accessing AI agents via an MCP server; AI agents can read or generate repository files, create branches, commit changes, trigger pipelines, and reclaim pipeline states through GitLab MCP. Use ArgoCD MCP to query application status, trigger synchronization, and read difference status. The system uses PostgreSQL MCP to write model registration information, image summary, model file path, service object identifier, and deployment status. Read pre-deployment capacity information and post-deployment acceptance metrics using Prometheus MCP; Kubectl provides local execution capabilities through built-in tools for direct deployment, status query, log diagnostics, and deployment acceptance. The system sets up the `model_deployment_registry` table in PostgreSQL, which stores the following fields: `model_name` (model name), `namespace` (namespace), `model_type` (model type), `deployment_mode` (deployment mode), `gitlab_project_path` (project path), `git_ref` (branch), `manifest_path` (manifest path), `image_ref` (image address), `image_digest` (image digest), `minio_bucket` (model bucket name), `minio_object_key` (model file path), `service_name` (service name), `configmap_name` (configuration object name), and `deploy_status` (deployment status), ensuring that model deployment results are queryable and traceable.

8. The AI ​​model deployment system based on AI intelligent agents according to claim 7, characterized in that, The deployment sequence for a simple test scenario deployment system includes: After the user submits the model deployment YAML template, service exposure template, Secret key object or ConfigMap configuration object template, target namespace, image address, model file path and resource parameters, the interaction and entry layer extracts the key information of InferenceService inference service object, Deployment workload object, Service service object and Secret key object from the template, and parses the target name, namespace, image field and resource field; Subsequently, the context and memory layer writes the template summary, user-added parameters, and target environment into the SQLite session storage. The core orchestration layer determines that the task is a test direct deployment task and selects the local executor toolchain. The AI ​​agent generates the final YAML manifest text based on the current parameters and calls the k8s_save_manifest tool to securely write the manifest to the isolated working directory. After the local executor returns the file path, the core orchestration layer calls the k8s_apply_manifest tool, which loads Kubeconfig and distributes resources via Kubectl. After the resource is created, the AI ​​agent checks the object status using k8s_list_resources and k8s_describe_resource; For model services based on KServe, the InferenceService status, service entry point, and key readiness conditions are also checked. If an anomaly occurs during deployment, the AI ​​agent calls k8s_get_events and k8s_get_logs to obtain the event stream and log fragments, and provides a diagnostic conclusion based on the results of previous executions. Subsequently, the context and memory layer writes the template summary, user-added parameters, and target environment to the SQLite session store. The core orchestration layer determines that the task is a test direct deployment task and selects the local executor toolchain. The AI ​​agent generates the final YAML manifest text based on the current parameters and calls the k8s_save_manifest tool to safely write the manifest to the isolated working directory. After the local executor returns the file path, the core orchestration layer calls the k8s_apply_manifest tool, which loads Kubeconfig and distributes resources via Kubectl. After the resource is created, the AI ​​agent checks the object status using k8s_list_resources and k8s_describe_resource; For model services based on KServe, the InferenceService status, service entry point, and key readiness conditions are also checked. If an anomaly occurs during deployment, the AI ​​agent calls k8s_get_events and k8s_get_logs to obtain the event stream and log fragments, and provides a diagnostic conclusion based on the results of previous executions. This path can complete template verification and model service trial deployment within a relatively short link, making it suitable as a verification entry point before standard pipeline deployment.

9. The AI ​​model deployment system based on AI intelligent agents according to claim 8, characterized in that, The standard production scenario pipeline deployment system provides standard pipeline deployment for production delivery scenarios. The standard pipeline deployment sequence includes seven stages: The first phase involves compiling deployment requirements; The second phase involves organizing warehouse templates; The third phase consists of image building, pushing, and inventory updating; The fourth phase is GitOps synchronous deployment; The fifth stage is cluster deployment and resource creation; The sixth stage is deployment and acceptance; The seventh stage involves deployment registration and presetting of operating parameters.

10. The AI ​​model deployment system based on AI agents according to claim 9, characterized in that: The deployment requirements include: extracting the model name, model type, inference framework, image base, model file source, target environment, whether to use Knative serverless mode, and resource constraint parameters from the AI ​​agent, and writing them into the session context; The repository template organization includes: the AI ​​agent reads the template repository through GitLab MCP and generates or completes the inference service code, Dockerfile container build script, KServe or Knative resource template, ArgoCDApplication application manifest and GitLab CI / CD pipeline configuration in the target project; Image building, pushing, and manifest updates include: GitLab Runner executes a pipeline in an isolated container of Docker Executor, first downloading the model weight file from MinIO, then combining the inference service code, Dockerfile, and model file to build the image, and pushing it to the GitLab Registry; after the pipeline is completed, it automatically updates the image tag in the Kubernetes YAML manifest and commits it to the configuration repository; GitOps synchronous deployment includes: GitLab notifies ArgoCD via Webhook, ArgoCD pulls the latest declarative manifest, compares the target state with the current cluster state, and synchronizes the changes to the Kubernetes cluster; Cluster deployment and resource creation include: Kubernetes nodes pull private images through pre-configured imagePullSecrets, and create workloads, service objects, and inference endpoints; Deployment acceptance includes: the AI ​​agent, combined with the local executor, Prometheus, and log query capabilities, checks the status of resource objects, Pod readiness status, endpoint health status, and minimal inference call results; For model services that comply with Open Inference Protocol V2, the system performs health checks, metadata reading, sample calls, and return result verification; for generative models, the system performs lightweight text generation verification through an OpenAI-style interface. Deployment registration and pre-setting of running parameters include: the AI ​​agent writes the model source, image address, image summary, MinIO object path, service object identifier and deployment status into PostgreSQL, and creates or updates the corresponding ConfigMap running configuration object, providing an entry point for monitoring, parameter tuning and elastic policy takeover after the model goes online.

Images