Large model system efficient operation maintenance method and device

Abstract

The invention discloses an efficient operation and maintenance method and device for a large model system, and belongs to the technical field of artificial intelligence, and the implementation of the method comprises the following steps: constructing an operation and maintenance agent cluster of the large model system, which comprises a model deployment agent, a model detection agent, a model operation agent, a model upgrading agent and an operation and maintenance planning agent; after an operation and maintenance task is input, the operation and maintenance planning agent receives an input instruction, performs task understanding and analysis, and divides the task into a conventional operation and maintenance mode and a customized operation and maintenance mode; setting a conventional operation and maintenance mode by using the operation and maintenance planning agent, and calling the model detection agent and the model operation agent to guarantee and maintain function indexes and operation states of models in the system; and setting a customized operation and maintenance mode by using the operation and maintenance planning agent, and calling the model deployment agent and the model upgrading agent to complete customized deployment and upgrading of the model. According to the method, efficient deployment, continuous verification, dynamic expansion and intelligent upgrading of the model are realized.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method and apparatus for the efficient operation and maintenance of large model systems. Background Technology

[0002] In recent years, thanks to the support of big data and enhanced computing power, artificial intelligence (AI) technology has made significant progress. AI is increasingly becoming a core technology leading a new round of technological revolution and industrial transformation, with its applications continuously being implemented across various fields, drastically changing existing production and lifestyles. The release of ChatGPT in 2022 marked the arrival of the era of generative AI large-scale models. These generative models excel in simulating human logical thinking and language processing abilities, and can more intelligently mimic human communication and thought processes, laying the foundation for the development of intelligent auxiliary tools and smart assistants. However, as large-scale model technologies and products are gradually applied across various industries, how to achieve efficient operation and maintenance of large-scale models to ensure that they can quickly respond to business needs while maintaining high levels of performance and stability during application is a critical challenge that urgently needs to be addressed in the practical application of large-scale models. Summary of the Invention

[0003] The technical objective of this invention is to address the above-mentioned shortcomings by providing a method and apparatus for the efficient operation and maintenance of a large model system. This method and apparatus can achieve efficient deployment, continuous verification, dynamic expansion, and intelligent upgrading of the model, ensuring that the large model can quickly respond to business needs and maintain a high level of performance and stability during application.

[0004] The technical solution adopted by this invention to solve its technical problem is: An efficient operation and maintenance method for large model systems is proposed, which achieves efficient operation and maintenance of large model systems based on multi-agent collaborative driving. The implementation of this method includes the following steps: S1. Construct a cluster of intelligent agents for the operation and maintenance of a large model system, including intelligent agents for model deployment, model detection, model operation, model upgrade, and operation and maintenance planning. S2. After the operation and maintenance task is input, the operation and maintenance planning intelligent agent receives the input instruction, performs task understanding and analysis, and divides the task into two types of modes: routine operation and maintenance and customized operation and maintenance. S3. Use the operation and maintenance planning agent to set up the regular operation and maintenance mode, and call the model detection agent and the model running agent to ensure and maintain the functional indicators and running status of the models in the system. S4. Use the operation and maintenance planning agent to set up a customized operation and maintenance mode, and call the model deployment agent and the model upgrade agent to complete the customized deployment and upgrade of the model.

[0005] This method constructs a cluster of intelligent agents for the operation and maintenance of large model systems by building intelligent agents for model deployment, model detection, model operation, model upgrade, and operation and maintenance planning. It provides intelligent operation and maintenance services for software systems with large models as their core, enabling efficient deployment, continuous verification, dynamic expansion, and intelligent upgrades of models. While ensuring that the models can quickly respond to business needs during application, it can also maintain a high level of functionality and stability, thus ensuring the stable application of large model products in actual engineering.

[0006] Furthermore, in step S1, the large model system operation and maintenance intelligent agent cluster is specifically as follows: The model deployment agent can call containerization packaging tools, pipeline engine tools, and environment consistency assurance tools to achieve unattended automated deployment of "submit and go live" through containerization and standardization, CI / CD pipeline; The model detection agent can call model function verification tools, real-time monitoring tools, and self-healing control tools to realize continuous verification and monitoring of model functions, and respond to model detection anomaly signals to perform self-healing operations including dynamic traffic switching and version rollback to ensure the stability of model functions. The model running agent can call metric collection tools, elastic scaling control tools, traffic management tools, and model lifecycle management tools to achieve dynamic expansion and load balancing of model services. Through model lifecycle management, it can perform model preloading when the replica instance starts and graceful exit when it terminates. The model upgrade agent can call multiple version deployment tools, network control tools, and decision execution tools to realize intelligent upgrades and dynamic iterations of the model, and establish a feedback-driven dynamic iteration mechanism to ensure that the model can be updated in a timely manner according to business needs and user feedback, maintaining its competitiveness and applicability. The operation and maintenance planning intelligent agent is used to realize the collaborative work among various intelligent agents, and has two modes: routine operation and maintenance and customized operation and maintenance.

[0007] Furthermore, in step S3, the specific steps of the conventional operation and maintenance mode include: S3.1: Send the model function detection task to the model detection agent and perform model function detection; S3.2: Send the model runtime status maintenance task to the model runtime agent and execute the model runtime status maintenance.

[0008] Furthermore, in step S3.1, the specific steps for performing the model function detection task include: S3.1.1: After receiving the model function detection task, the model detection agent first calls the model function verification tool to perform verification periodically before and after model deployment to ensure the model's functional correctness. The verification content includes: Input validation checks whether the input data meets expectations. Output validation: Check whether the model output meets expectations; Business logic validation verifies whether the model output conforms to business rules and allows users to customize settings according to their own business rules. Model performance testing, including testing the model's response time and throughput; Adversarial testing uses adversarial examples to test the robustness of the model; Compliance testing checks whether the model has any bias, discrimination, or violations. S3.1.2: The model detection agent invokes a real-time monitoring tool to monitor the model's operational status in the runtime environment. The monitoring content includes: Input data drift is monitored using statistical tests to determine if the distribution of the input data has changed. Output data drift; monitor changes in the distribution of the model output. Model performance metrics, including the accuracy and recall of the detection model; System metrics include monitoring system response time, system CPU and memory resource usage. System anomaly detection: Use anomaly detection models to detect anomalies in inputs, outputs, or intermediate results. S3.1.3: During the execution of steps S3.1.1 and S3.1.2, the model detection agent simultaneously invokes the self-healing control tool to perform self-healing operations on abnormal signals during the detection process, including: Dynamic traffic switching operations route requests to the backup model; Version rollback operation, loading a historical stable model version.

[0009] Furthermore, in step S3.2, the specific steps for performing the model running state maintenance task include: S3.2.1: After receiving the model running status maintenance task, the model running agent first encapsulates the model as a containerized service, creates multiple replica instances of the model service in the container orchestration platform, calls the indicator collection tool, and collects the resource utilization indicators and custom business indicators of each replica instance of the model service. The resource utilization includes CPU utilization and memory utilization, and the custom business indicators are indicators customized by the user according to the specific business needs of the model. S3.2.2: Then the model running agent calls the elastic scaling control tool to dynamically adjust the number of model service replica instances based on the collected resource utilization indicators and custom business indicators compared with the set indicator thresholds. S3.2.3: Afterwards, the model running agent calls the traffic management tool to configure the load balancing service. Specifically, during the scaling up process, newly created replica instances are automatically added to the backend instance pool; during the scaling down process, new requests are stopped from being distributed to replica instances to be terminated, and they are removed from the backend instance pool after the existing requests have been processed. S3.2.4: Finally, the model running agent calls the model lifecycle management tool to manage the entire lifecycle of the model service, performs model preloading during the model service startup phase, and performs graceful exit during the model service termination phase.

[0010] Furthermore, in step S4, the specific steps of the customized operation and maintenance mode include: S4.1: Receive customized tasks input by the user, break down and classify the tasks, send the model deployment tasks to the model deployment agent, and execute the model deployment tasks; S4.2: Send the model upgrade task to the model upgrade agent and execute the model upgrade task.

[0011] Furthermore, in step S4.1, the specific steps for executing the model deployment task include: S4.1.1: After receiving the model deployment task, the model deployment agent first calls the containerization encapsulation tool to define the directory structure of the model file, the dependency list format and the startup script interface, and builds a separate training dependency and inference runtime through the container orchestration platform to achieve multi-environment deployment; S4.1.2: Then the model deployment agent calls the pipeline engine tool to design a multi-stage CI / CD pipeline, and executes unit testing, security scanning, image building and smoke verification in sequence; S4.1.3: After the model is deployed, call the environment consistency assurance tool, dynamically adjust resource allocation according to custom indicators, and monitor the environment consistency drift in real time.

[0012] Furthermore, in step S4.2, the specific steps for performing the model upgrade task include: S4.2.1: After receiving the model upgrade task, the model upgrade agent first calls the multi-version deployment tool to deploy multiple version model services in the Kubernetes cluster, with each version assigned an independent version label; S4.2.2: Then the model upgrade agent calls the network control tool, which distributes the traffic according to the set model upgrade weight value to different version model services by controlling the configuration traffic. The weight is gradually adjusted to gradually transfer the traffic to the upgrade version and complete the gray-scale upgrade of the model. S4.2.3: During the model upgrade process, the model upgrade agent calls the decision execution tool to dynamically adjust the model upgrade weight value by judging the performance indicators of the new version model service reported by the user. When the performance indicators of the model service are higher than the set threshold, the model upgrade weight is automatically increased and more traffic is directed to the new version model. When the performance indicators are lower than the set threshold, the model upgrade weight is gradually reduced. When the weight reaches zero, the model service version is rolled back to the old version.

[0013] This invention also claims a device for efficient operation and maintenance of a large model system, the system comprising: a user interaction module, an intelligent agent cluster suite, an operation and maintenance toolset, and a visualization module; The user interaction module is used for interaction between users and the system, including: inputting operation and maintenance tasks, setting business indicators, and inputting customized business requirements; The Intelligent Agent Cluster Suite is used to build an intelligent agent cluster for the operation and maintenance of large storage model systems, including model deployment intelligent agents, model detection intelligent agents, model running intelligent agents, model upgrade intelligent agents, and operation and maintenance planning intelligent agents; it also provides customization building functions, allowing users to customize the construction of intelligent agents. The operations and maintenance toolset is used to build and aggregate operations and maintenance tools called by intelligent agents, including: containerization packaging tools, pipeline engine tools, environment consistency assurance tools, indicator collection tools, elastic scaling control tools, traffic management tools, model lifecycle management tools, multi-version deployment tools, network control tools, and decision execution tools. It also provides interfaces for calling third-party tools and user-customized tools. The visualization module provides a visual interactive interface for displaying operation and maintenance results and for visual interaction with customers. This device can achieve efficient operation and maintenance of large model systems through the methods described above.

[0014] The present invention also claims a computer-readable medium storing computer instructions that, when executed by a processor, enable the implementation of the above-described method.

[0015] Compared with the prior art, the efficient operation and maintenance method and apparatus for a large-scale model system of the present invention has the following advantages: This invention constructs a cluster of intelligent agents for the operation and maintenance of large model systems by building intelligent agents for model deployment, model detection, model operation, model upgrade, and operation and maintenance planning. It provides intelligent operation and maintenance services for software systems with large models as their core, enabling efficient deployment, continuous verification, dynamic expansion, and intelligent upgrades of models. While ensuring that the models can quickly respond to business needs during application, they can also maintain a high level of functionality and stability, thus ensuring the stable application of large model products in actual engineering and facilitating the application of artificial intelligence large model technology in various industries. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating an efficient operation and maintenance method for a large model system according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating the efficient operation and maintenance task execution process of a large model system provided in one embodiment of the present invention. Detailed Implementation

[0017] The present invention will be further described below with reference to specific embodiments.

[0018] This invention provides a method for efficient operation and maintenance of large model systems, based on multi-agent collaborative driving to achieve efficient operation and maintenance of large model systems. For example... Figure 1 As shown, this method first constructs a cluster of intelligent agents for large-scale model system operations and maintenance, including a model deployment agent, a model detection agent, a model operation agent, a model upgrade agent, and an operation and maintenance planning agent. Then, after an operation and maintenance task is input, the operation and maintenance planning agent receives the input instructions, understands and analyzes the task, and categorizes the task into two modes: routine operation and maintenance and customized operation and maintenance. In the routine operation and maintenance mode, the operation and maintenance planning agent calls the model detection agent and the model operation agent to ensure and maintain the functional indicators and operational status of the models in the system. In the customized operation and maintenance mode, the operation and maintenance planning agent calls the model deployment agent and the model upgrade agent to complete the customized deployment and upgrade of the models. Figure 2 As shown, under the scheduling of the operation and maintenance planning intelligent agent, the collaborative work of each intelligent agent is realized, and the operation and maintenance tasks are completed by calling tools, thereby achieving efficient deployment, continuous verification, dynamic expansion and intelligent upgrading of the model.

[0019] The specific steps for implementing this method are as follows: Step 1: Construct a large-scale model system operation and maintenance intelligent agent cluster, including model deployment intelligent agents, model detection intelligent agents, model operation intelligent agents, model upgrade intelligent agents, and operation and maintenance planning intelligent agents. Specifically: The model deployment agent can call containerization packaging tools, pipeline engine tools, and environment consistency assurance tools to achieve unattended automated deployment of "submit and go live" through containerization and standardization, and CI / CD pipeline.

[0020] The model detection agent can call model function verification tools, real-time monitoring tools, and self-healing control tools to achieve continuous verification and monitoring of model functions. It can also respond to abnormal model detection signals by performing a series of self-healing operations such as dynamic traffic switching and version rollback to ensure the stability of model functions.

[0021] The model running agent can call metric collection tools, elastic scaling control tools, traffic management tools, and model lifecycle management tools to achieve dynamic expansion and load balancing of model services. Through model lifecycle management, it can perform model preloading when the replica instance starts and graceful exit when it terminates.

[0022] The model upgrade agent can call upon multiple versions of deployment tools, network control tools, and decision execution tools to achieve intelligent upgrades and dynamic iterations of the model. It also establishes a feedback-driven dynamic iteration mechanism to ensure that the model can be updated in a timely manner according to business needs and user feedback, maintaining its competitiveness and applicability.

[0023] The operation and maintenance planning intelligent agent is used to realize the collaborative work among various intelligent agents, and has two modes: routine operation and maintenance and customized operation and maintenance.

[0024] Step 2: After the operation and maintenance task is input, the operation and maintenance planning intelligent agent receives the input instructions, understands and analyzes the task, and divides the task into two types: routine operation and maintenance and customized operation and maintenance.

[0025] Step 3: Utilize the operation and maintenance planning agent to set up a routine operation and maintenance mode, and invoke the model detection agent and model operation agent to ensure and maintain the functional indicators and operational status of the models in the system. The specific steps are as follows: Step 3.1: Send the model function detection task to the model detection agent and perform model function detection, specifically as follows: Step 3.1.1: After receiving the model function detection task, the model detection agent first calls the model function verification tool to periodically verify the model before and after deployment to ensure the model's functional correctness. The verification content specifically includes: (1) Input validation: Check whether the input data meets expectations, such as data type, range, and handling of missing values; (2) Output validation: Check whether the model output meets expectations, such as output range, data type, etc. (3) Business logic verification: verify whether the model output conforms to business rules, such as whether the sum of the probabilities of each category is 1 in a classification task, and support users to make custom settings according to their own business rules. (4) Model performance testing, testing the model's response time, throughput, and other performance characteristics; (5) Adversarial testing: using adversarial examples to test the robustness of the model; (6) Compliance testing: check whether the model has any bias, discrimination, or violations.

[0026] Step 3.1.2: Then, the model detection agent calls a real-time monitoring tool to monitor the model's running status in the runtime environment. The monitoring content specifically includes: (1) Input data drift: Use statistical test methods such as KS test to monitor whether the distribution of input data has changed; (2) Output data drift, monitor the distribution changes of the model output; (3) Model performance metrics, such as accuracy and recall of the detection model; (4) System indicators, monitoring system response time, system CPU, memory and other resource usage; (5) System anomaly detection: Use anomaly detection model to detect anomalies in inputs, outputs or intermediate results.

[0027] Step 3.1.3: During the execution of steps 3.1.1 and 3.1.2, the model detection agent simultaneously invokes the self-healing control tool to perform self-healing operations on abnormal signals during the detection process, including: (1) Dynamic traffic switching operation, which routes requests to the backup model; (2) Version rollback operation, loading historical stable model versions.

[0028] Step 3.2: Send the model runtime status maintenance task to the model runtime agent to perform model runtime status maintenance, specifically as follows: Step 3.2.1: After receiving the model running status maintenance task, the model running agent first encapsulates the model as a containerized service, creates multiple replica instances of the model service in the container orchestration platform, calls the indicator collection tool, and collects the resource utilization indicators and custom business indicators of each replica instance of the model service. Resource utilization includes CPU utilization, memory utilization, etc., and custom business indicators are indicators customized by users according to the specific business needs of model use.

[0029] Step 3.2.2: Then the model running agent calls the elastic scaling control tool to dynamically adjust the number of model service replica instances based on the collected resource utilization indicators and custom business indicators compared with the set indicator thresholds.

[0030] Step 3.2.3: Then the model runs the agent to call the traffic management tool to configure the load balancing service. Specifically, during the scaling up process, newly created replica instances are automatically added to the backend instance pool; during the scaling down process, new requests are stopped from being distributed to replica instances to be terminated, and they are removed from the backend instance pool after the existing requests have been processed.

[0031] Step 3.2.4: Then the model running agent calls the model lifecycle management tool to manage the entire lifecycle of the model service, performs model preloading during the model service startup phase, and performs graceful exit during the model service termination phase.

[0032] Step 4: Utilize the operation and maintenance planning agent to set up a customized operation and maintenance mode, and call the model deployment agent and model upgrade agent to complete the customized deployment and upgrade of the model. The specific steps are as follows: Step 4.1: Receive customized tasks input by the user, break down and classify the tasks, send the model deployment tasks to the model deployment agent, and execute the model deployment tasks, specifically: Step 4.1.1: After receiving the model deployment task, the model deployment agent first calls the containerization encapsulation tool to define the directory structure of the model files, the dependency list format and the startup script interface, and builds a separate training dependency and inference runtime through the container orchestration platform to achieve multi-environment deployment.

[0033] Step 4.1.2: Then the model deployment agent calls the pipeline engine tool to design a multi-stage CI / CD pipeline, and executes unit testing, security scanning, image building and smoke verification in sequence.

[0034] Step 4.1.3: After the model is deployed, call the environment consistency assurance tool, dynamically adjust resource allocation according to custom indicators, and monitor the environment consistency drift in real time.

[0035] Step 4.2: Send the model upgrade task to the model upgrade agent and execute the model upgrade task, specifically as follows: Step 4.2.1: After receiving the model upgrade task, the model upgrade agent first calls the multi-version deployment tool to deploy multi-version model services in the Kubernetes cluster, with each version assigned an independent version label.

[0036] Step 4.2.2: Then the model upgrade agent calls the network control tool, which distributes traffic proportionally to different version model services based on the set model upgrade weight value, and gradually adjusts the weight to gradually transfer traffic to the upgrade version, thus completing the gray-scale upgrade of the model.

[0037] Step 4.2.3: During the model upgrade process, the model upgrade intelligence will invoke the decision execution tool to dynamically adjust the model upgrade weight value based on the performance indicators of the new version model service reported by users. For example, when the performance indicators of the model service are higher than the set threshold, the model upgrade weight will be automatically increased, and more traffic will flow to the new version model. When the performance indicators are lower than the set threshold, the traffic weight will be gradually reduced. When the weight reaches zero, the model service version will be rolled back to the old version.

[0038] This method addresses the operation and maintenance of software systems with large models at their core. It constructs a cluster of intelligent agents for large model system operation and maintenance, including model deployment agents, model detection agents, model execution agents, model upgrade agents, and operation and maintenance planning agents. The model deployment agent utilizes containerization tools, pipeline engine tools, and environment consistency assurance tools to achieve automated and efficient model deployment. The model detection agent utilizes model function verification tools, real-time monitoring tools, and self-healing control tools to continuously verify and monitor model functions, ensuring model stability. The model execution agent utilizes indicator collection tools, elastic scaling control tools, traffic management tools, and model lifecycle management tools to achieve dynamic expansion and load balancing of model services. The model upgrade agent utilizes multi-version deployment tools, network control tools, and decision execution tools to achieve intelligent upgrades and dynamic iterations of models. The operation and maintenance planning agent enables collaborative work among the agents, offering both routine and customized operation and maintenance modes. Based on this method, and driven by the multi-agent collaboration of the large model system operation and maintenance agent cluster, a highly intelligent, scalable, secure, and controllable large model system operation and maintenance environment can be created.

[0039] This invention also provides an efficient operation and maintenance device for large model systems based on multi-agent collaborative driving. This device can achieve efficient operation and maintenance of large model systems through the efficient operation and maintenance method for large model systems based on multi-agent collaborative driving described in the above embodiments. The system includes: a user interaction module, an agent cluster suite, an operation and maintenance toolset, and a visualization module.

[0040] The user interaction module is used for interaction between users and the system, including: inputting operation and maintenance tasks, setting business indicators, and inputting customized business requirements.

[0041] The Intelligent Agent Cluster Suite is used to build intelligent agent clusters for the operation and maintenance of large-scale storage model systems. It includes model deployment agents, model detection agents, model running agents, model upgrade agents, and operation and maintenance planning agents. Customization capabilities are also provided, allowing users to customize the construction of these agents.

[0042] The operations and maintenance toolset is used to build and aggregate operations and maintenance tools for intelligent agents, including: containerization packaging tools, pipeline engine tools, environment consistency assurance tools, indicator collection tools, elastic scaling control tools, traffic management tools, model lifecycle management tools, multi-version deployment tools, network control tools, and decision execution tools. It also provides interfaces for calling third-party tools and user-customized tools.

[0043] The visualization module provides a visual interactive interface for displaying operation and maintenance results and for visual interaction with customers.

[0044] This invention also provides a computer-readable medium storing computer instructions. When executed by a processor, these instructions implement the efficient operation and maintenance method for a large model system based on multi-agent cooperative driving described in the above embodiments. Specifically, a system or apparatus equipped with a storage medium storing software program code that implements the functions of any of the embodiments described above can be provided, enabling the computer (or CPU or MPU) of the system or apparatus to read and execute the program code stored in the storage medium.

[0045] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of the present invention.

[0046] Storage media embodiments for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0047] Furthermore, it should be clear that not only can the program code read by the computer be executed, but also the operating system or other components operating on the computer can be instructed based on the program code to perform some or all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0048] Furthermore, it is understood that the program code read from the storage medium is written to the memory set in the expansion board inserted into the computer or to the memory set in the expansion unit connected to the computer. Then, based on the instructions of the program code, the CPU or other components installed on the expansion board or expansion unit execute some and all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0049] The present invention has been shown and described in detail above with reference to the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above embodiments, those skilled in the art will know that more embodiments of the present invention can be obtained by combining the code review methods in the different embodiments. These embodiments are also within the protection scope of the present invention.

Claims

1. A method for efficient operation and maintenance of a large-scale model system, characterized in that, The method for achieving efficient operation and maintenance of large model systems based on multi-agent collaborative driving includes the following steps: S1. Construct a cluster of intelligent agents for the operation and maintenance of a large model system, including intelligent agents for model deployment, model detection, model operation, model upgrade, and operation and maintenance planning. S2. After the operation and maintenance task is input, the operation and maintenance planning intelligent agent receives the input instruction, performs task understanding and analysis, and divides the task into two types of modes: routine operation and maintenance and customized operation and maintenance. S3. Use the operation and maintenance planning agent to set up the regular operation and maintenance mode, and call the model detection agent and the model running agent to ensure and maintain the functional indicators and running status of the models in the system. S4. Use the operation and maintenance planning agent to set up a customized operation and maintenance mode, and call the model deployment agent and the model upgrade agent to complete the customized deployment and upgrade of the model.

2. The method for efficient operation and maintenance of a large model system according to claim 1, characterized in that, In step S1, the large model system operation and maintenance intelligent agent cluster is specifically as follows: The model deployment agent can call containerization packaging tools, pipeline engine tools, and environment consistency assurance tools to achieve unattended automated deployment of "submit and go live" through containerization and standardization, CI / CD pipelines; The model detection agent can call model function verification tools, real-time monitoring tools, and self-healing control tools to realize continuous verification and monitoring of model functions, and respond to model detection anomaly signals to perform self-healing operations including dynamic traffic switching and version rollback to ensure the stability of model functions. The model running agent can call metric collection tools, elastic scaling control tools, traffic management tools, and model lifecycle management tools to achieve dynamic expansion and load balancing of model services. Through model lifecycle management, it can perform model preloading when the replica instance starts and exit when it terminates. The model upgrade agent can call multiple version deployment tools, network control tools, and decision execution tools to realize intelligent upgrades and dynamic iterations of the model, and establish a feedback-driven dynamic iteration mechanism to ensure that the model can be updated in a timely manner according to business needs and user feedback, maintaining its competitiveness and applicability. The operation and maintenance planning intelligent agent is used to realize the collaborative work among various intelligent agents, and has two modes: routine operation and maintenance and customized operation and maintenance.

3. The efficient operation and maintenance method for a large model system according to claim 1, characterized in that, In step S3, the specific steps of the conventional operation and maintenance mode include: S3.1: Send the model function detection task to the model detection agent and perform model function detection; S3.2: Send the model runtime status maintenance task to the model runtime agent and execute the model runtime status maintenance.

4. The efficient operation and maintenance method for a large model system according to claim 3, characterized in that, In step S3.1, the specific steps for performing the model function detection task include: S3.1.1: After receiving the model function detection task, the model detection agent first calls the model function verification tool to perform verification periodically before and after model deployment to ensure the model's functional correctness. The verification content includes: Input validation checks whether the input data meets expectations. Output validation: Check whether the model output meets expectations; Business logic validation verifies whether the model output conforms to business rules and allows users to customize settings according to their own business rules. Model performance testing, including testing the model's response time and throughput; Adversarial testing uses adversarial examples to test the robustness of the model; Compliance testing checks whether the model has any bias, discrimination, or violations. S3.1.2: The model detection agent invokes a real-time monitoring tool to monitor the model's operational status in the runtime environment. The monitoring content includes: Input data drift is monitored using statistical tests to determine if the distribution of the input data has changed. Output data drift; monitor changes in the distribution of the model output. Model performance metrics, including the accuracy and recall of the detection model; System metrics include monitoring system response time, system CPU and memory resource usage. System anomaly detection: Use anomaly detection models to detect anomalies in inputs, outputs, or intermediate results. S3.1.3: During the execution of steps S3.1.1 and S3.1.2, the model detection agent simultaneously invokes the self-healing control tool to perform self-healing operations on abnormal signals during the detection process, including: Dynamic traffic switching operations route requests to the backup model; Version rollback operation, loading a historical stable model version.

5. The method for efficient operation and maintenance of a large model system according to claim 3, characterized in that, In step S3.2, the specific steps for performing the model running state maintenance task include: S3.2.1: After receiving the model running status maintenance task, the model running agent first encapsulates the model as a containerized service, creates multiple replica instances of the model service in the container orchestration platform, calls the indicator collection tool, and collects the resource utilization indicators and custom business indicators of each replica instance of the model service. The resource utilization includes CPU utilization and memory utilization, and the custom business indicators are indicators customized by the user according to the specific business needs of the model. S3.2.2: Then the model running agent calls the elastic scaling control tool to dynamically adjust the number of model service replica instances based on the collected resource utilization indicators and custom business indicators compared with the set indicator thresholds. S3.2.3: Afterwards, the model running agent calls the traffic management tool to configure the load balancing service. Specifically, during the scaling up process, newly created replica instances are automatically added to the backend instance pool; during the scaling down process, new requests are stopped from being distributed to replica instances to be terminated, and they are removed from the backend instance pool after the existing requests have been processed. S3.2.4: Finally, the model running agent calls the model lifecycle management tool to manage the entire lifecycle of the model service. It performs model preloading during the model service startup phase and exits during the model service termination phase.

6. The efficient operation and maintenance method for a large model system according to claim 1, characterized in that, In step S4, the specific steps of the customized operation and maintenance mode include: S4.1: Receive customized tasks input by the user, break down and classify the tasks, send the model deployment tasks to the model deployment agent, and execute the model deployment tasks; S4.2: Send the model upgrade task to the model upgrade agent and execute the model upgrade task.

7. The method for efficient operation and maintenance of a large model system according to claim 6, characterized in that, In step S4.1, the specific steps for executing the model deployment task include: S4.1.1: After receiving the model deployment task, the model deployment agent first calls the containerization encapsulation tool to define the directory structure of the model file, the dependency list format and the startup script interface, and builds a separate training dependency and inference runtime through the container orchestration platform to achieve multi-environment deployment; S4.1.2: Then the model deployment agent calls the pipeline engine tool to design a multi-stage CI / CD pipeline, and executes unit testing, security scanning, image building and smoke verification in sequence; S4.1.3: After the model is deployed, call the environment consistency assurance tool, dynamically adjust resource allocation according to custom indicators, and monitor the environment consistency drift in real time.

8. The efficient operation and maintenance method for a large model system according to claim 6, characterized in that, In step S4.2, the specific steps for performing the model upgrade task include: S4.2.1: After receiving the model upgrade task, the model upgrade agent first calls the multi-version deployment tool to deploy multiple version model services in the Kubernetes cluster, with each version assigned an independent version label; S4.2.2: Then the model upgrade agent calls the network control tool, which distributes the traffic according to the set model upgrade weight value to different version model services by controlling the configuration traffic. The weight is gradually adjusted to gradually transfer the traffic to the upgrade version and complete the gray-scale upgrade of the model. S4.2.3: During the model upgrade process, the model upgrade agent calls the decision execution tool to dynamically adjust the model upgrade weight value by judging the performance indicators of the new version model service reported by the user. When the performance indicators of the model service are higher than the set threshold, the model upgrade weight is automatically increased and traffic is directed to the new version model. When the performance indicators are lower than the set threshold, the model upgrade weight is gradually reduced. When the weight reaches zero, the model service version is rolled back to the old version.

9. A high-efficiency operation and maintenance device for a large model system, characterized in that, The system includes: a user interaction module, an intelligent agent cluster suite, an operation and maintenance toolset, and a visualization module; The user interaction module is used for interaction between users and the system, including: inputting operation and maintenance tasks, setting business indicators, and inputting customized business requirements; The Intelligent Agent Cluster Suite is used to build an intelligent agent cluster for the operation and maintenance of large storage model systems, including model deployment intelligent agents, model detection intelligent agents, model running intelligent agents, model upgrade intelligent agents, and operation and maintenance planning intelligent agents; it also provides customization building functions, allowing users to customize the construction of intelligent agents. The operations and maintenance toolset is used to build and aggregate operations and maintenance tools called by intelligent agents, including: containerization packaging tools, pipeline engine tools, environment consistency assurance tools, indicator collection tools, elastic scaling control tools, traffic management tools, model lifecycle management tools, multi-version deployment tools, network control tools, and decision execution tools. It also provides interfaces for calling third-party tools and user-customized tools. The visualization module provides a visual interactive interface for displaying operation and maintenance results and for visual interaction with customers. This device can achieve efficient operation and maintenance of large model systems through the method described in any one of claims 1 to 8.

10. A computer-readable medium, characterized in that, The computer-readable medium stores computer instructions that, when executed by a processor, enable the implementation of the method described in any one of claims 1 to 8.

Images