Large model all-in-one machine batch filling system and method

Abstract

The invention relates to the technical field of large-model all-in-one machine deployment, in particular to a large-model all-in-one machine batch filling system and method, and the system comprises a GPU server, a network switch, a deployment server, a deployment medium, and a deployment service. The method has the beneficial effects that unattended deployment of a large model and an intelligent agent platform is realized, and automatic deployment can be realized only by filling in deployment configuration list information and uploading a configuration file to a deployment server by deployment personnel. Automatic starting of service startup is supported, and the all-in-one machine can be used immediately after being opened. The large model deployment threshold is reduced, the deployment efficiency is improved, and the batch deployment cost of deliverymen is saved.

Description

Technical Field

[0001] This invention relates to the field of large model integrated machine deployment technology, specifically a system and method for batch filling of large model integrated machines. Background Technology

[0002] In November 2022, OpenAI, a renowned American AI research company, released ChatGPT, an AI chatbot program based on the large language model GPT-3.5. Its fluent language expression, powerful question-handling capabilities, and massive database garnered widespread attention worldwide. Various industries felt the powerful impact of large models. Simultaneously, a research boom in large models officially began both domestically and internationally, with DeepSeek becoming particularly popular due to its technological innovation, cost advantages, open-source ecosystem, market demand resonance, and symbolic significance of a breakthrough in domestic AI. The continuous growth in the delivery volume of large model products, particularly the hundreds or thousands of large model all-in-one machines, severely impacted project delivery schedules due to traditional manual deployment methods.

[0003] To simplify the deployment of large-scale integrated machines, lower the deployment threshold, and improve the efficiency of batch deployment, this invention provides a device and method for batch filling of large-scale integrated machines. Summary of the Invention

[0004] The purpose of this invention is to provide a system and method for batch filling of large-scale integrated models, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a system for batch filling of large-scale integrated models, comprising:

[0006] GPU server: Supports different GPU models, including Huawei Ascend NPU, Tianshu Zhixin GPU, Muxi GPU, and NVIDIA GPU; Operating system support includes Ubuntu, openEuler, and Kylin; Supports different CPU architectures such as X86_64, Aarch64, and C86;

[0007] Network switches: used to enable network connections and data transmission between various devices;

[0008] Deployment server: Runs deployment and DHCP services, and stores model files, GPU drivers, agents, OCR services, and Kubernetes media packages; the system supports Windows, Linux, and Mac; the architecture supports x86_64, c86, and AArch; the hardware can be a laptop, edge box, or server;

[0009] Deployment media: Includes K8S deployment media, agent deployment media, large model file media, model image files, and GPU drivers; large model files include DeepSeek-R1 full version, DeepSeek-R1-Distill-Llama-70B, DeepSeek-R1-Distill-Qwen-32B, Qwen2.5-72B-Instruct, bge-m3, and bge-reranker-v2-m3; running model images include vllm, sglang, mindie, and xinference startup frameworks;

[0010] Deployment Service: Based on the configuration file, the model service and intelligent agent are deployed. The service runs in a containerized manner through the Kubernetes environment, supporting large model all-in-one machines with different GPU models, different CPU architectures and different operating systems, and realizing unattended automatic deployment.

[0011] Preferably, the deployment service supports component selection deployment, specifically: models, GPU drivers, Kubernetes, and agent components can be selected for deployment and installation; it supports uploading media packages to the large model all-in-one machine, and uploading and deploying specified components in an unattended manner according to the configuration.

[0012] Preferably, the system supports simultaneous loading of large model servers in batches. Specifically, it uses a deployment service to uniformly configure and manage multiple large model all-in-one machines, enabling the simultaneous loading of models, agents, and related components onto multiple large model all-in-one machines, thereby improving loading efficiency.

[0013] Preferably, the system includes a startup service template, specifically: the startup service template is a Helmchart template, which supports the configuration of necessary startup service parameters, including model file name, model authentication key, maximum value of model token, model GPU card number, number of GPU parallel operations, and number of parallel clients.

[0014] Preferably, the Kubernetes in the system has the following functions: support for creating Kubernetes clusters, adding and deleting nodes, creating container images, and high availability; support for automatic startup of model and agent applications, enabling large model all-in-one machines to be used out of the box.

[0015] A method for batch filling of large-scale integrated models includes the following steps:

[0016] Equipment and Media Preparation: Prepare a GPU server that supports different GPU models, including Huawei Ascend NPU, Tianshu Zhixin GPU, Muxi GPU, and NVIDIA GPU; operating system support includes Ubuntu, openEuler, and Kylin; and support for different CPU architectures such as X86_64, Aarch64, and C86.

[0017] Prepare a network switch to enable network connections and data transmission between devices;

[0018] Prepare a server for deployment, running deployment and DHCP services, and storing model files, GPU drivers, agents, OCR services, and Kubernetes media packages; the system supports Windows, Linux, and Mac; the architecture supports x86_64, c86, and AArch; the hardware can be a laptop, edge box, or server;

[0019] Prepare the deployment media, including K8S deployment media, agent deployment media, large model file media, model image files, and GPU drivers; the large model files include DeepSeek-R1 full version, DeepSeek-R1-Distill-Llama-70B, DeepSeek-R1-Distill-Qwen-32B, Qwen2.5-72B-Instruct, bge-m3, and bge-reranker-v2-m3; the running model images include vllm, sglang, mindie, and xinference startup frameworks;

[0020] Configure and deploy services: Configure deployment services on the deployment server so that it can deploy model services and agents according to the configuration file. The service runs in a containerized manner through the Kubernetes environment to support large model all-in-one machines with different GPU models, different CPU architectures and different operating systems, and realize unattended automatic deployment.

[0021] Startup service template configuration: Use the Helmchart template as the startup service template and configure the necessary parameters for starting the service, such as model file name, model authentication key, maximum model token value, model GPU card number, GPU parallelism count, and number of parallel clients;

[0022] Kubernetes cluster configuration: Configure Kubernetes to support Kubernetes cluster creation, adding and deleting nodes, creating container images, high availability, and automatic startup of model and agent applications.

[0023] Preferably, the step of deploying the model service and the agent according to the configuration file further includes:

[0024] Read configuration file: The deployment service reads the pre-set configuration file to obtain the deployment parameter information of the model service and intelligent agent components;

[0025] Component selection and deployment: Based on the selections in the configuration file, deploy and install the model, GPU driver, Kubernetes, and agent components; support uploading media packages to the large model all-in-one machine, and upload and deploy specified components in an unattended manner according to the configuration;

[0026] Containerized deployment: Components are deployed in a containerized manner through the Kubernetes environment, ensuring stable operation on large-scale all-in-one machines with different GPU models, CPU architectures, and operating systems.

[0027] Preferably, the steps for simultaneously loading large batches of models onto servers are as follows:

[0028] Batch server identification: The deployment service identifies multiple large model servers that need to be loaded simultaneously and obtains their hardware and software configuration information;

[0029] Unified configuration management: Based on a unified configuration file, deployment parameters for multiple large model servers can be set, including model selection and component installation options;

[0030] Parallel filling operation: The deployment service sends filling instructions to multiple large model servers simultaneously, performing filling operations on models, agents and related components in parallel, thereby improving filling efficiency.

[0031] Preferably, the steps to support automatic startup of model and agent applications upon system boot are as follows:

[0032] Startup configuration: Configure the startup parameters for models and agent applications in Kubernetes so that they are automatically loaded when the system starts;

[0033] Dependency service check: Before the application starts, check whether the required dependent services, such as the Kubernetes cluster and GPU driver, are running normally;

[0034] Automatic startup execution: When dependent services are running normally and the system meets the startup conditions, the model and agent application are started automatically, making the large model all-in-one machine ready to use out of the box.

[0035] Preferably, the following optimization steps are also included:

[0036] Deployment process monitoring: During the deployment service operation, monitor the deployment progress and status in real time, and record log information during the deployment process;

[0037] Error handling and recovery: When errors occur during deployment, appropriate actions are taken based on the error type, such as re-uploading the media package or reconfiguring parameters; for recoverable errors, automatic recovery operations are performed to ensure that the deployment process continues.

[0038] Deployment result verification: After deployment, the functions of the large model all-in-one machine are verified, and the model service and intelligent agent components are checked to ensure the quality and stability of batch filling.

[0039] Compared with the prior art, the beneficial effects of the present invention are:

[0040] The system and method for batch loading of large-scale models using an integrated machine proposed in this invention enables unattended deployment of large-scale models and intelligent agent platforms. Deployment personnel only need to fill in the deployment configuration list information and upload the configuration file to the deployment server to achieve automatic deployment. It supports automatic service startup, making the integrated machine ready to use out of the box. This lowers the barrier to deployment of large-scale models, improves deployment efficiency, and saves on batch deployment costs for delivery personnel. Attached Figure Description

[0041] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the present invention clear and complete, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only some, not all, embodiments of the present invention, and are merely illustrative of the embodiments of the present invention. They are not intended to limit the embodiments of the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] Example 1: This invention provides a technical solution: a system for batch filling of large-scale integrated models, comprising:

[0044] 1. Deploy the server

[0045] The deployment server stores deployment media and runs deployment services. It connects to the same network switch as the large-scale all-in-one machine's GPU server and stores deployment media in a specific directory on the server. The deployment server can be various hardware devices such as laptops, edge boxes, and servers. Supported systems include Windows, Linux, and Mac, and supported architectures include c86, x86_64, and aarch64. The deployment service automatically executes deployment tasks through deployment configuration files and supports batch installation of different component services onto large-scale all-in-one machines.

[0046] 2. GPU server

[0047] The large-scale model appliance is a GPU server for deploying large-scale models and intelligent agent applications. Depending on the requirements, it includes GPUs from different manufacturers and models, such as NVIDIA, Huawei Ascend NPU, Tianshu Zhixin TianGai GPU, and MuXi GPU. The large-scale model appliance needs to be rack-mounted and its system and data disks configured with RAID. Then, the operating system is installed and the network is configured. Afterwards, the deployment service issues deployment tasks to install and deploy the large-scale model and intelligent agent applications.

[0048] 3. Deployment Media

[0049] The media includes model files such as DeepSeek-R1, DeepSeek-R1-Distill-Llama-70B, Qwen2.5-72B-Instruct, bge-m3, bge-reranker-v2-m3, etc.

[0050] The media includes GPU driver packages, which contain installers for drivers or firmware from different GPU manufacturers, the k8s-device-plugin, container images provided by the manufacturers, driver dependency packages, installation scripts, and Helm chart templates for model startup. The GPU driver packages include driver installation packages for Huawei Ascend, Tianshu Zhixin, Muxi, and NVIDIA, as well as Huawei Ascend firmware installation packages. Manufacturer model container image startup frameworks include vllm, sglang, mindie, and xinference, etc., which are loaded and pushed to the Kubernetes environment image repository during deployment. Driver dependency packages contain dependency libraries such as gcc required for driver installation. The k8s-device-plugin contains plugins required for using the GPU in the Kubernetes environment. Installation scripts include driver installation scripts, Kubernetes device plugin installation scripts, and model deployment scripts. The Helm chart template is a template file for starting the model, rendered by the Helm tool into a YAML resource file directly supported by Kubernetes, which is then applied to the Kubernetes environment for model startup. Helmchart templates support defining startup model parameters, such as model file name, model authentication key, maximum model token value, model GPU card number, GPU parallelism count, and number of parallel clients.

[0051] The media includes Kubernetes installation packages, including the Sealos installation tool, Kubernetes, Calico network components, Helm commands, etc., used to create Kubernetes clusters and container image repositories.

[0052] The media package includes an agent application platform that supports integration with different models and the creation and orchestration of agents. The application platform media package contains service components such as Helmchart templates, service container images, and deployment scripts. The Helmchart template is the template file for starting the agent platform application. It is rendered by the Helm tool into a YAML resource file directly supported by Kubernetes, and the YAML resource file is applied to the Kubernetes environment to start the platform service. The Helmchart template supports the definition of agent startup parameters, such as database address, database user, agent port, mount path, etc.

[0053] The media includes an OCR service installation package, which contains a Helmchart template, container image, installation scripts, etc. Deploying the OCR service requires a GPU card.

[0054] The media contains a deployment service package, which includes deployment services, deployment configuration, and installation scripts, for batch installation of large-scale all-in-one machines.

[0055] 4. Deployment Service

[0056] On the deployment server, the deployment service media is unzipped and the installation script is executed. The service is managed by systemd and runs continuously. A deployment configuration file is created in YAML format. Configuration parameters include global configurations for the local installation media package path, remote storage media path, GPU driver package, Kubernetes media package, model file package, agent package, k8s-device-plugin package, and OCR service package. Each media package can be configured with a media processing mode, supporting modes such as upload only, install only, upload and install media, and no operation. Configuration parameters also include a host information array containing information about the all-in-one machine hosts where large models need to be deployed, including IP address, username, password, SSH port, host CPU architecture, GPU model, and an array of components to exclude from installation (GPU driver, agent, OCR service, Kubernetes, and model). Configuring excluded components skips their installation. After creating and modifying the configuration file, it is placed in the designated processing directory for the deployment service. The service automatically reads the configuration and performs the deployment operation. After deployment, the configuration file is moved to the deployment directory. Configuration files can also be uploaded via the simple web interface provided by the deployment service to create deployment tasks. Progress and results can be viewed through the interface or in the deployment service's backend logs. The configuration file format is as follows:

[0057]

[0058]

[0059]

[0060]

[0061] Example 2, based on Example 1, proposes a method for batch filling of a large-scale integrated machine according to claim 5, comprising the following steps:

[0062] 1. Switch preparation: Install and power on the switch, and configure the local area network VLAN.

[0063] 2. Deploy the server, connect it to the switch, and after powering on, automatically start the deployment service and DHCP service, and write the deployment server IP into the configuration file to set a fixed IP address.

[0064] 3. Place the large-scale integrated machine to be filled on the rack, connect it to the switch, power it on, and obtain the IP address via DHCP service.

[0065] 4. Create a deployment file and write parameters such as the IP address of the all-in-one machine to be filled, SSH port, user, password, CPU architecture, GPU model, local path of the deployment media package, deployment media package, remote storage path of the deployment media package, deployment component information, and deployment model.

[0066] 5. Upload the deployment configuration file through the deployment service interface or the deployment server directory.

[0067] 6. The deployment service discovers the deployment configuration file and triggers the deployment operation. It reads the deployment file information, iterates through the appliances to be deployed, and creates a coroutine for each appliance to execute the deployment task.

[0068] 7. Deploy the service by writing the IP address of the all-in-one machine into the configuration file to fix it. For example, for Ubuntu systems, write it into the file / etc / netplan / 50-cloud-init.yaml and stop the cloud-init service on the host to prevent the host from automatically updating the IP address. For openEuler systems, write it into the network card file in the / etc / sysconfig / network-scripts directory.

[0069] 8. The deployment service determines whether to upload the GPU media package to the all-in-one machine. If so, it copies the local GPU media package to the specified directory on the all-in-one machine according to the host's GPU model. Otherwise, proceed to the next step.

[0070] 9. The deployment service determines whether to install the GPU media package to the all-in-one machine. If so, it executes the installation script within the GPU media package based on the host's GPU model. The script automatically installs driver packages such as gcc, drivers, and firmware, and returns a restart status code depending on the situation. If the script execution result returns a restart status code, the deployment service restarts the all-in-one machine and waits for the all-in-one machine to start before proceeding to the next step. If the GPU media package is not installed, proceed directly to the next step.

[0071] 10. Deployment service determines whether to upload the Kubernetes media package to the appliance. If so, copy the corresponding Kubernetes media package from the host's CPU architecture to the specified directory on the appliance. Otherwise, proceed to the next step.

[0072] 11. Deployment Service: Determine if the Kubernetes media package needs to be installed on the appliance. If so, execute the installation script within the Kubernetes media package on the appliance. The script automatically installs Kubernetes and the container image repository service. If Kubernetes is not installed, proceed to the next step.

[0073] 12. The deployment service checks whether the k8s-device-plugin media package has been uploaded to the all-in-one machine. If so, it copies the corresponding k8s-device-plugin media package from the local GPU driver package to the specified GPU driver directory on the all-in-one machine, based on the host's GPU model. Otherwise, proceed to the next step.

[0074] 13. The deployment process checks if the k8s-device-plugin media package is installed on the all-in-one machine. If so, it executes the installation script within the k8s-device-plugin media package. The script automatically installs the runtime media package provided by the GPU manufacturer, configures the containerd configuration file for the k8s component, restarts the containerd service, and deploys the k8s-device-plugin service pod. If the k8s-device-plugin is not installed, proceed directly to the next step.

[0075] 14. The deployment service determines whether to upload the model file media package to the all-in-one machine. If so, it copies the local model file media package to the specified directory on the all-in-one machine according to the configuration. Otherwise, proceed to the next step.

[0076] 15. The deployment service determines whether to deploy the model to the appliance. If so, it executes the model installation script from the appliance's GPU media package. The script automatically imports the image and pushes it to the image repository, renders the Helm chart template, and applies it to the Kubernetes environment. The model file is mounted and run via pods, and the model's pod number is specified in the Helm chart template configuration file. If the model is not being deployed, proceed directly to the next step.

[0077] 16. The deployment service determines whether to upload the OCR file media package to the all-in-one machine. If so, it copies the local OCR media package to the specified directory on the all-in-one machine according to the host GPU model. Otherwise, proceed to the next step.

[0078] 17. Deployment Service: Determine whether to deploy OCR to the appliance. If so, execute the installation script within the appliance's OCR media package. The script automatically imports the image and pushes it to the image repository, renders the Helm chart template, and applies it to the Kubernetes environment, running via pods. The model's pod number is specified through the Helm chart template configuration file. If not deploying an OCR service, proceed directly to the next step.

[0079] 18. The deployment service determines whether to upload the intelligent agent file media package to the all-in-one machine. If so, it copies the local intelligent agent media package to the specified directory on the all-in-one machine according to the host's CPU architecture. Otherwise, proceed to the next step.

[0080] 19. Deploy Service: Determine whether to deploy the agent service to the appliance. If yes, execute the installation script within the agent service media package on the appliance. The script automatically imports the image and pushes it to the image repository, renders the Helmchart template, and applies it to the Kubernetes environment, running via pods. If not deploying the agent service, proceed directly to the next step.

[0081] 20. The deployment service interface displays the deployment task progress status. After successful deployment, reinstall the program to modify the IP address and then shut down the large model all-in-one machine. Move the deployment configuration file from the deployment directory to the deployment completion directory, indicating successful deployment. For configuration files that failed to deploy, move them to the pending directory, manually investigate the cause, fix it, and then re-execute the deployment task. For erroneous tasks, the execution steps will be recorded on the large model all-in-one machine host; after fixing it, resume the deployment from where it failed.

[0082] 21. After the large-scale model all-in-one machine is successfully deployed and shut down, it is taken off the shelf and sent to the project site. The project site is powered on and the host is logged into to configure the machine IP. The IP modification program is executed, and the large-scale model application of the all-in-one machine runs normally.

[0083] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A system for batch filling of large-scale integrated models, characterized in that: include: GPU server: Supports different GPU models, including Huawei Ascend NPU, Tianshu Zhixin GPU, Muxi GPU, and NVIDIA GPU; Operating system support includes Ubuntu, openEuler, and Kylin; Supports different CPU architectures such as X86_64, Aarch64, and C86; Network switches: used to enable network connections and data transmission between various devices; Deployment server: Runs deployment and DHCP services, and stores model files, GPU drivers, agents, OCR services, and Kubernetes media packages; the system supports Windows, Linux, and Mac. The architecture supports x86_64, c86, and aarch; the hardware can be in the form of laptops, edge boxes, or servers. Deployment media: Includes K8S deployment media, agent deployment media, large model file media, model image files, and GPU drivers; large model files include DeepSeek-R1 full version, DeepSeek-R1-Distill-Llama-70B, DeepSeek-R1-Distill-Qwen-32B, Qwen2.5-72B-Instruct, bge-m3, and bge-reranker-v2-m3; running model images include vllm, sglang, mindie, and xinference startup frameworks; Deployment Service: Based on the configuration file, the model service and intelligent agent are deployed. The service runs in a containerized manner through the Kubernetes environment, supporting large model all-in-one machines with different GPU models, different CPU architectures and different operating systems, and realizing unattended automatic deployment.

2. The system for batch filling of large-scale integrated models according to claim 1, characterized in that: The deployment service supports component selection deployment, specifically: you can choose to deploy and install models, GPU drivers, Kubernetes, and intelligent agent components; it supports uploading media packages to the large model all-in-one machine, and uploading and deploying specified components in an unattended manner according to the configuration.

3. The system for batch filling of large-scale integrated models according to claim 2, characterized in that: The system supports simultaneous loading of large model servers in batches. Specifically, it deploys services to uniformly configure and manage multiple large model all-in-one machines, enabling the simultaneous loading of models, agents, and related components onto multiple large model all-in-one machines, thereby improving loading efficiency.

4. The system for batch filling of large-scale integrated models according to claim 3, characterized in that: The system includes a startup service template, specifically a Helmchart template, which supports the configuration of necessary startup service parameters, including model file name, model authentication key, maximum model token value, model GPU card number, GPU parallelism count, and number of parallel clients.

5. The system for batch filling of large-scale integrated models according to claim 4, characterized in that: The Kubernetes system has the following features: it supports Kubernetes cluster creation, adding and deleting nodes, creating container images, and high availability; it supports automatic startup of model and agent applications, enabling large model all-in-one machines to be used out of the box.

6. A method for batch filling of a large-scale integrated model machine according to claim 5, characterized in that: Includes the following steps: Equipment and Media Preparation: Prepare a GPU server that supports different GPU models, including Huawei Ascend NPU, Tianshu Zhixin GPU, Muxi GPU, and NVIDIA GPU; operating system support includes Ubuntu, openEuler, and Kylin; and support for different CPU architectures such as X86_64, Aarch64, and C86. Prepare a network switch to enable network connections and data transmission between devices; Prepare a server for deployment, running deployment and DHCP services, and storing model files, GPU drivers, agents, OCR services, and Kubernetes media packages; the system supports Windows, Linux, and Mac. The architecture supports x86_64, c86, and aarch; the hardware can be in the form of laptops, edge boxes, or servers. Prepare the deployment media, including K8S deployment media, agent deployment media, large model file media, model image files, and GPU drivers; the large model files include DeepSeek-R1 full version, DeepSeek-R1-Distill-Llama-70B, DeepSeek-R1-Distill-Qwen-32B, Qwen2.5-72B-Instruct, bge-m3, and bge-reranker-v2-m3; the running model images include vllm, sglang, mindie, and xinference startup frameworks; Configure and deploy services: Configure deployment services on the deployment server so that it can deploy model services and agents according to the configuration file. The service runs in a containerized manner through the Kubernetes environment to support large model all-in-one machines with different GPU models, different CPU architectures and different operating systems, and realize unattended automatic deployment. Startup service template configuration: Use the Helmchart template as the startup service template and configure the necessary parameters for starting the service, such as model file name, model authentication key, maximum model token value, model GPU card number, GPU parallelism count, and number of parallel clients; Kubernetes cluster configuration: Configure Kubernetes to support Kubernetes cluster creation, adding and deleting nodes, creating container images, high availability, and automatic startup of model and agent applications.

7. A method according to claim 6, characterized in that: The steps for deploying the model service and the agent according to the configuration file further include: Read configuration file: The deployment service reads the pre-set configuration file to obtain the deployment parameter information of the model service and intelligent agent components; Component selection and deployment: Based on the selections in the configuration file, deploy and install the model, GPU driver, Kubernetes, and agent components; support uploading media packages to the large model all-in-one machine, and upload and deploy specified components in an unattended manner according to the configuration; Containerized deployment: Components are deployed in a containerized manner through the Kubernetes environment, ensuring stable operation on large-scale all-in-one machines with different GPU models, CPU architectures, and operating systems.

8. A method according to claim 7, characterized in that: The steps to support simultaneous batch loading of large models on servers are as follows: Batch server identification: The deployment service identifies multiple large model servers that need to be loaded simultaneously and obtains their hardware and software configuration information; Unified configuration management: Based on a unified configuration file, deployment parameters for multiple large model servers can be set, including model selection and component installation options; Parallel filling operation: The deployment service sends filling instructions to multiple large model servers simultaneously, performing filling operations on models, agents and related components in parallel, thereby improving filling efficiency.

9. A method according to claim 8, characterized in that: The steps to support automatic startup of model and agent applications are as follows: Startup configuration: Configure the startup parameters for models and agent applications in Kubernetes so that they are automatically loaded when the system starts; Dependency service check: Before the application starts, check whether the required dependent services, such as the Kubernetes cluster and GPU driver, are running normally; Automatic startup execution: When dependent services are running normally and the system meets the startup conditions, the model and agent application are started automatically, making the large model all-in-one machine ready to use out of the box.

10. A method according to claim 9, characterized in that: The following optimization steps are also included: Deployment process monitoring: During the deployment service operation, monitor the deployment progress and status in real time, and record log information during the deployment process; Error handling and recovery: When errors occur during deployment, appropriate actions are taken based on the error type, such as re-uploading the media package or reconfiguring parameters; For recoverable errors, automatic recovery operations are performed to ensure that the deployment process continues; Deployment result verification: After deployment, the functions of the large model all-in-one machine are verified, and the model service and intelligent agent components are checked to ensure the quality and stability of batch filling.

Images