Degradation and disaster recovery system for large language model agent application

Abstract

The invention discloses a degradation and disaster recovery system for large language model agent application, which relates to the technical field of large model application, and comprises a model service layer used for deploying a model, managing according to a model instance, and deploying a cloud model as a multi-level backup and elastic expansion pool; the workflow arrangement layer is used for arrangement of business logic and multi-model and multi-step calling; the application service layer is a business unified entry, is deployed in Kubernetes and is used for routing, priority scheduling and result aggregation; the API gateway layer is used for unifying a gateway to be responsible for authentication, current limiting, fusing and model routing; the monitoring layer is used for collecting Prometheus format indexes and business indexes; and the strategy engine layer is a centralized strategy execution unit and is used for receiving the monitoring alarm and issuing a control instruction. According to the method, the blank of large model application in disaster recovery and degradation is filled up, and the problem of high availability of the large model application is solved with relatively low cost.

Description

Technical Field

[0001] This invention relates to the field of large model application technology, specifically to a degradation and disaster recovery system for large language model intelligent agent applications. Background Technology

[0002] Kubernetes: A rich open-source container orchestration system that achieves disaster recovery through multi-replica, multi-availability zone deployment and etcd backup, and ensures the stability of core services through resource limits, QoS, HPA, and mesh circuit breaking for degradation. Disadvantages: Kubernetes is too heavyweight for small-scale scenarios; its support for GPUs is limited, GPU scheduling is complex and inefficient, and large models are slow to load and cold start due to the need to load several gigabytes of model weights; Kubernetes cannot manage non-container-deployed components, such as purchased cloud products.

[0003] Currently, large-scale model applications face many challenges in terms of high availability and stability, including disaster recovery and degradation, for the following reasons: Large-scale model output is uncertain, making the traditional "HTTP 200+ latency" SLA unsuitable; different business scenarios have different latency requirements, such as intelligent customer service scenarios requiring high response speeds, while user lead mining scenarios have less stringent latency requirements; large-scale models are expensive, and computing resources are typically limited. High availability usually means multiple replicas constantly running on GPUs, drastically increasing costs. Direct degradation, while saving resources, also impacts user experience; large-scale model applications typically include workflow orchestration systems, such as the open-source orchestration system Dify used in this invention, as well as n8n, LangChain, etc. These workflow orchestration systems usually consist of multiple serial or parallel nodes. The longer the workflow and the more services involved (LLM, vector library, database, external API), the more the overall availability decreases, and failure in any link can lead to the interruption of the entire process. Summary of the Invention

[0004] To fill the gap in disaster recovery and degradation for large model applications, the purpose of this invention is to provide a degradation and disaster recovery system for large language model intelligent agent applications, aiming to solve the high availability problem of large model applications at a lower cost.

[0005] To achieve the above technical objectives, this application provides a degradation and disaster recovery system for large language model intelligent agent applications, including: The model service layer is used to deploy models and manage them by model instance. At the same time, cloud models are deployed as multi-level backup and elastic scaling pools. Workflow orchestration layer, used for orchestrating business logic and multi-model, multi-step calls; The application service layer serves as a unified entry point for business operations and is deployed on Kubernetes for routing, priority scheduling, and result aggregation. The API gateway layer is used to unify the gateway's responsibilities for authentication, rate limiting, circuit breaking, and model routing. The monitoring layer is used to collect Prometheus-formatted metrics and business metrics. The policy engine layer is a centralized policy execution unit used to receive monitoring alarms and issue control commands. Policies are stored in the form of rule sets and priority matrices, and support auditing and rollback operations.

[0006] Preferably, the model service layer is also used to report metrics in Prometheus format via the enable-metrics startup parameter of vllm, including the first token time and the total request duration.

[0007] Preferably, the workflow orchestration layer consists of several independent instances, supports cold standby and hot standby strategies, and exposes a standardized REST API. The workflow orchestrator provides multi-instance deployment, with each critical business workflow having at least two running instances configured in the application service layer (primary / backup), adding retries to the call steps of the large model, and reporting Prometheus-formatted metrics through open-source libraries of various programming languages.

[0008] Preferably, the application service layer is also used to provide a management backend to configure application-workflow-model mapping and application priorities. Applications are classified based on business tag metrics, and priority ranking takes effect when services are reassigned due to anomalies. When the strategy engine issues a scheduling instruction that the system is approaching its capacity limit and resources need to be reallocated, the application service layer is responsible for reclaiming low-priority applications, performing soft shutdown, and releasing resources. At the same time, it reports performance metrics in Prometheus format and business metrics of each application through an open-source library.

[0009] Preferably, the API gateway layer is also used to implement request interception, policy distribution interfaces, and health awareness and rapid switching of downstream services. Specifically, it provides API keys to business parties that call application services, sets different call volume quotas based on the API keys, and distributes requests to different model services by percentage according to the actions issued by the policy engine.

[0010] Preferably, the monitoring layer is also used for metric collection, health checks, anomaly detection, and alarm rule configuration. When an anomaly is detected, the monitoring layer calls the policy engine's API and reports the original metrics, event context, and timestamp; at the same time, it issues an alarm; and it continues to report the recovery status until the problem is closed.

[0011] Preferably, the strategy engine layer is also used to check whether the same event has been processed before execution to avoid repeated execution; and to write the execution status to the database so that it can be audited and rolled back. At the same time, each strategy issuance is accompanied by the decision reason, triggering conditions, execution actions and expected effects, which are recorded in the audit log and notified to the operation and maintenance developers.

[0012] Preferably, the API gateway layer also provides passive protection for degradation / disaster recovery strategies; Passive protection includes tiered rate limiting, used to implement multi-layered rate limiting strategies at the API gateway layer. Applications are rate-limited by API Key; model rate limiting protects individual model instances by configuring TPM rate limiting for each model caller; downstream model / workflow calls use a circuit breaker mode, configuring error rate and latency thresholds, returning a default degradation result or triggering backup logic during circuit breaker operation to reduce the risk of cascading failures; model degradation configures model fallback policies at the API gateway layer, allowing requests to be directly forwarded to cloud models when self-deployed model calls fail.

[0013] Preferably, the strategy engine layer provides proactive responses for degradation / disaster recovery strategies, including: a. For model unavailability: The triggering condition is N consecutive health check failures or the error rate / timeout threshold being exceeded, or a model-level DOWN alarm; the actions to be performed include: routing all requests within the affected area to the cloud pool; marking the model as maintenance mode and writing it to the event database; continuously monitoring the model's recovery status; if recovery fails, triggering an alarm to notify operations and maintenance; the recovery logic is that after the monitoring layer reports UP, wait for a stable window, and then return traffic to the self-deployed system according to a phased rollback strategy; b. Regarding model performance degradation: The trigger condition is that the P95 index of TTFT / RT exceeds the threshold, and the threshold is set according to different models; the execution actions include calling the gateway POST / gateway / flowcontrol / model to update the model routing weights; the back-switch logic is a phased back-switch, and the stability and error rate are verified at each step; c. For workflow orchestration layer anomalies: The triggering conditions are an increase in the error rate of the primary instance, excessively long task queues, or failed health checks; The action is to switch the request to the backup instance by calling the application service layer's POST / app / workflowcontrol / {wokflowId} interface through the strategy engine; The rollback logic is a phased rollback, with stability and error rate verified at each step. d. Assurance for business metrics driven by performance indicators: The trigger condition is that the SLI of critical business is lower than the SLO; the actions include calling GET / app / flowcontrol / rules to obtain the application priority configuration, reducing or pausing low-priority applications, and adjusting the application quota by calling POST / app / flowcontrol / {appId} to close the quota of low-priority applications; the rollback logic is a phased rollback, with the quota of low-priority applications decreasing by 10% in each step, and the stability of high-priority applications is verified at each step.

[0014] Preferably, the strategy engine layer also includes strategy priority, conflict resolution, and idempotency, wherein, The strategy priority mechanism assigns a priority field to each strategy. When an event is triggered, the strategy engine evaluates the applicable strategies in descending order of priority. When a mutually exclusive strategy is encountered, a conflict resolution strategy is used.

[0015] The above-described technical solution of the present invention brings about the following technical effects: Multi-model pool degradation strategy: Define "hot pool" (self-deployed model) and "cloud pool" (cloud model) at the same time and switch automatically through real-time monitoring indicators. It also supports proportional distribution rather than full switching. Hot pool is charged based on computing power and only related to usage time, while cloud pool is charged based on the number of calls. No call is charged, resulting in lower total cost.

[0016] Cross-layer disaster recovery orchestration: The strategy engine can trigger degradation / switch actions at the model service layer, workflow orchestration layer and application service layer simultaneously, and has conflict resolution and idempotency mechanisms.

[0017] Phased rollback mechanism: After recovery, traffic is rolled back proportionally, and stability is verified at each step. This is a more robust design than the traditional "immediate rollback".

[0018] Priority-driven business assurance: Under high load or abnormal conditions, low-priority application resources are dynamically reduced based on business priorities to ensure the normal use of critical business operations.

[0019] Unified monitoring and policy interface: Metrics are uniformly formatted using Prometheus, and the policy engine issues commands through a standardized API, forming a loosely coupled architecture. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1This is a flowchart of the system strategy execution described in this invention. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0023] It should be noted that the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or device.

[0024] like Figure 1 As shown, this invention provides a degradation and disaster recovery system for large language model intelligent agent applications, including: ModelLayer: Self-deployed models are preferred. In this embodiment, efficient inference frameworks such as VLLM can be used for optional management by model instance. The cloud model adopts a third-party cloud model as a multi-level backup and elastic scaling pool.

[0025] Workflow Layer: Responsible for orchestrating business logic and multi-model, multi-step calls. It consists of several independent instances and supports cold and hot standby strategies. It exposes a standardized REST API.

[0026] Application Layer: The unified entry point for business logic, deployed on Kubernetes, responsible for routing, priority scheduling, and result aggregation. It provides a management backend for configuring application-workflow-model mappings and application priorities.

[0027] API Gateway Layer: The unified gateway is responsible for authentication, rate limiting, circuit breaking, and model routing. The gateway implements request interception, policy distribution interfaces, and health awareness and rapid switching of downstream services.

[0028] The monitoring layer (Observability) uniformly collects Prometheus-formatted metrics and business metrics. This includes metric collection, health checks, anomaly detection, and alarm rule configuration.

[0029] Policy Engine Layer: A centralized policy execution unit that receives monitoring alerts and issues control commands (such as model switching, traffic allocation, workflow switching, and resource reclamation). Policies are stored in the form of rule sets and priority matrices, supporting auditing and rollback operations.

[0030] Specifically, the detailed design of each layer is as follows: (1) Model Service Layer: Provides multi-level model pools, including hot pools and cloud pools. The hot pools are characterized by self-deployment, multiple replicas, and low latency. The cloud pools are third-party cloud models of the same model, billed on a per-call basis, and serve as external backup / elastic capacity. Prometheus-formatted metrics, including TTFT (Time to First Token) and RT (Total Request Duration), are reported via the `--enable-metrics` startup parameter of vllm.

[0031] (2) Workflow orchestration layer: The workflow orchestrator provides multi-instance deployment, with each critical business workflow having at least two running instances configured at the application service layer, one primary and one backup. This includes adding retries to the call steps of large models and reporting Prometheus-formatted metrics through open-source libraries in various programming languages.

[0032] (3) Application service layer: Priority calculation: Applications are classified based on business tag indicators, i.e., priority level. The priority ranking takes effect when services are reassigned due to anomalies.

[0033] Resource reclamation: When the strategy engine issues a scheduling instruction that the system is approaching its capacity limit and resources need to be reallocated, the application service layer is responsible for reclamating low-priority applications. At this time, the quota is reset to 0, soft shutdown is performed, and resources are released.

[0034] The system reports performance metrics in Prometheus format and business metrics for each application, such as response time (RT) and success / failure, through an open-source library.

[0035] The relevant APIs for the strategy are as follows: GET / app / flowcontrol / rules - Retrieves application priority configuration; POST / app / flowcontrol / {appId} - Temporarily adjust application quotas; POST / app / workflowcontrol / {wokflowId} - Switch between primary and backup workflows.

[0036] (4) API Gateway Layer: The main functions include authentication, rate limiting, and routing. Rate limiting is based on the API key provided by the business party, and routing includes model routing and application service layer routing.

[0037] The application service layer routing strategy is to provide an API Key to the business party calling the application service, and set different call volume quotas based on the API Key, such as QPS and QPM metrics; the model routing strategy is that the gateway distributes requests to different model services according to a percentage based on the actions issued by the strategy engine.

[0038] The relevant APIs for the strategy are as follows: POST / gateway / flowcontrol / model - Update model route weights.

[0039] (5) Monitoring layer: The main indicators used in this system for indicator collection are as follows: Model metrics: TTFT (P95), RT (P95); Workflow metrics: RT (P99), error rate; Application service layer metrics: RT (P99) and error rate for each application.

[0040] Health checks include health check endpoints such as / health or / ready provided by all three layers of services; health check results include: UP (normal), DEGRADED (degraded but still working), and DOWN (unavailable); monitoring logic uses sliding window detection, for example, triggering an alarm after N consecutive failures.

[0041] The alerting and notification process includes the following: when an anomaly is detected, the monitoring layer calls the policy engine's API and reports the raw metrics, event context, and timestamp; simultaneously, an alert is sent via email or instant messaging. Monitoring continues to report the recovery status until the issue is closed.

[0042] (6) Strategy Engine Layer: A strategy in the strategy model consists of a trigger, an action, a priority, an idempotent identifier (id), and a rollback strategy. Strategies are stored in a rule table and support versioning.

[0043] Action types include model route switching and percentage adjustment (self-deployment → cloud service), workflow instance switching (primary → backup), and application quota adjustment.

[0044] Idempotency and deduplication include checking whether the same event has already been processed based on the event ID or hash before policy execution to avoid duplicate execution. The policy execution status is written to the database and can be audited and rolled back.

[0045] Decision interpretability includes providing the rationale, triggering conditions, execution actions, and expected effects for each policy issuance, recording these in the audit log, and notifying the operations and maintenance developers.

[0046] The degradation / disaster recovery strategy described in this embodiment is divided into passive protection and active response. Passive protection specifically includes: Layered rate limiting involves implementing multi-layered rate limiting strategies at the API gateway layer. Applications are rate-limited based on API keys. Model rate limiting protects individual model instances by configuring TPM (Total Token Management) for each model caller, where TPM is the number of tokens generated per minute. Downstream model / workflow calls are handled using a circuit breaker mode, with configured error rates and latency thresholds. During circuit breaker operation, a default degradation result is returned or backup logic is triggered to reduce the risk of cascading failures.

[0047] Model degradation refers to configuring a fallback strategy for models at the API gateway layer, allowing requests to be forwarded directly to the cloud model when a self-deployed model call fails. Proactive response refers to proactive strategies that rely on monitoring triggers and a policy priority matrix, ordered from high to low priority. Typical strategies are as follows: (1) For model unavailable (model_down) The trigger condition is N consecutive times, where N can be 3, a health check failure, an error rate / timeout threshold being exceeded, or a model-level DOWN alarm.

[0048] The actions performed include: routing all requests within the affected area to the cloud pool; marking the model as maintenance mode and writing it to the event database, including timestamps and context; continuously monitoring the model's recovery status; and triggering an alarm to notify operations and maintenance if recovery fails.

[0049] The recovery logic is as follows: after the monitoring layer reports UP, wait for a stable window. If there is no DOWN report within 5 minutes, then switch traffic back to the self-deployed system according to the phased rollback strategy.

[0050] (2) Addressing model performance degradation (model_degrade) The trigger condition is that TTFT / RTP95 exceeds the threshold, and the threshold is set according to different models.

[0051] The actions performed include calling the gateway POST / gateway / flowcontrol / model to update the model route weights. Level 1 (Minor): Traffic is switched to cloud pool or downgraded model in segments of 10%-20%; at the same time, more self-deployed instances are enabled or the capacity is expanded.

[0052] Level 2 (Medium): Increase traffic splitting to 50% and downgrade non-critical functions (e.g., disable high-cost plugins).

[0053] Level 3 (Severe): Full switchover and triggering operation and maintenance intervention.

[0054] The back-cut logic is a phased back-cut, for example, it can step by 10% speed, and verify the stability and error rate at each step.

[0055] (3) Addressing workflow orchestration layer anomalies (workflow_down) The trigger conditions are an increase in the primary instance's error rate, excessively long task queues, or a failed health check. The actions performed include switching the request to the backup instance via a POST call to the application service layer's `POST / app / workflowcontrol / {wokflowId}` interface through the strategy engine. The rollback logic is phased, for example, in 10% increments, with stability and error rate verified at each step.

[0056] (4) Business-driven protection The trigger condition is that the SLI of a critical business, such as the success rate or response time, is lower than the SLO.

[0057] The actions performed include: calling GET / app / flowcontrol / rules to retrieve application priority configuration, reducing or pausing low-priority applications, and adjusting application quotas by calling POST / app / flowcontrol / {appId} to disable quotas for low-priority applications.

[0058] The rollback logic is a phased rollback, with low-priority applications' quotas decreasing by 10% in increments, and the stability of high-priority applications is verified at each step.

[0059] This embodiment also includes policy priority, conflict resolution, and idempotency. The policy priority mechanism includes: each policy has a `priority` field, which is an integer; the smaller the value, the higher the priority. When an event is triggered, the policy engine evaluates the applicable policies in descending order of priority. When a mutually exclusive policy is encountered, the conflict resolution strategy is used as follows: When two policies should both be executed according to the conditions but their actions conflict with each other, priority is used; if the priorities are the same, timestamps are used, and execution is based on the principle of first-come, first-served; if a decision still cannot be made, the authoritative scope of the policy is used, where system level > platform level > application level; the decision is written to the audit log and the operations and maintenance personnel are notified to intervene manually; if the decision is made automatically, it is marked as unacceptable.

[0060] Idempotency and deduplication: Policy execution is identified by a unique event ID, and the `executions` table is queried before each action to avoid duplicate execution. All changes are guaranteed to have eventual consistency through locks.

[0061] This invention provides a degradation and disaster recovery architecture, aiming to ensure that the system can monitor, make decisions, degrade, switch, and recover when large model instances are abnormal, workflow orchestration or external API pressure / failure occurs. It provides controllable availability and experience degradation solutions at different levels (large model, workflow orchestration service, external API), and supports hybrid deployment and scheduling of large model self-deployment and cloud services.

[0062] This invention addresses the problem that large language models often rely on a single inference service, primarily self-deployed or cloud-based. In such cases, if the model crashes, experiences performance degradation, or encounters network anomalies, the entire application may be interrupted. Alternatively, when model performance degrades, resulting in high latency and error rates, the fault can spread along the entire call chain, slowing down workflow orchestration and upstream applications, ultimately impacting the overall availability of the business.

[0063] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A degradation and disaster recovery system for large language model intelligent agent applications, characterized in that, include: The model service layer is used to deploy models and manage them by model instance. At the same time, cloud models are deployed as multi-level backup and elastic scaling pools. Workflow orchestration layer, used for orchestrating business logic and multi-model, multi-step calls; The application service layer serves as a unified entry point for business operations and is deployed on Kubernetes for routing, priority scheduling, and result aggregation. The API gateway layer is used to unify the gateway's responsibilities for authentication, rate limiting, circuit breaking, and model routing. The monitoring layer is used to collect Prometheus-formatted metrics and business metrics. The policy engine layer is a centralized policy execution unit used to receive monitoring alarms and issue control commands. Policies are stored in the form of rule sets and priority matrices, and support auditing and rollback operations.

2. The degradation and disaster recovery system for a large language model intelligent agent application according to claim 1, characterized in that: The model service layer also reports metrics in Prometheus format through the enable-metrics startup parameter of vllm, including the first token time and the total request duration.

3. The degradation and disaster recovery system for a large language model intelligent agent application according to claim 1, characterized in that: The workflow orchestration layer consists of several independent instances, supports cold standby and hot standby strategies, and exposes a standardized REST API. The workflow orchestrator provides multi-instance deployment, with each critical business workflow having at least two running instances configured in the application service layer (primary / backup). It also adds retries to the call steps of the large model and reports Prometheus-formatted metrics through open-source libraries of various programming languages.

4. The degradation and disaster recovery system for a large language model intelligent agent application according to claim 1, characterized in that: The application service layer is also used to provide a management backend to configure application-workflow-model mapping and application priorities. Applications are classified based on business tag metrics, and priority ranking takes effect when services are reassigned due to anomalies. When the strategy engine issues a scheduling instruction that the system is approaching its capacity limit and resources need to be reallocated, the application service layer is responsible for reclaiming low-priority applications, performing soft shutdown, and releasing resources. At the same time, it reports performance metrics in Prometheus format and business metrics of each application through an open-source library.

5. The degradation and disaster recovery system for a large language model intelligent agent application according to claim 1, characterized in that: The API gateway layer is also used to implement request interception, policy distribution interfaces, and health awareness and rapid switching of downstream services. Specifically, it provides API keys to business parties that call application services, sets different call volume quotas based on the API keys, and distributes requests to different model services according to a percentage based on the actions issued by the policy engine.

6. The degradation and disaster recovery system for a large language model intelligent agent application according to claim 1, characterized in that: The monitoring layer is also used for metric collection, health checks, anomaly detection, and alarm rule configuration. When an anomaly is detected, the monitoring layer calls the policy engine's API and reports the original metrics, event context, and timestamp; it also issues an alarm and continuously reports the recovery status until the problem is closed.

7. The degradation and disaster recovery system for a large language model intelligent agent application according to claim 1, characterized in that: The strategy engine layer is also used to check whether the same event has been processed before execution to avoid repeated execution; and to write the execution status to the database so that it can be audited and rolled back. At the same time, each strategy issuance is accompanied by the decision reason, triggering conditions, execution actions and expected effects, which are recorded in the audit log and notified to the operation and maintenance developers.

8. The degradation and disaster recovery system for a large language model intelligent agent application according to claim 1, characterized in that: The API gateway layer is also used to provide passive protection for degradation / disaster recovery strategies; the passive protection includes layered rate limiting, used to implement multi-layered rate limiting strategies at the API gateway layer, wherein... Applications should implement rate limiting based on API Key; Model rate limiting protects individual model instances by configuring TPM rate limiting for each model caller. The downstream model / workflow is called using the circuit breaker mode, with error rate and latency threshold configured. During the circuit breaker opening period, the default degradation result is returned or backup logic is triggered to reduce the risk of cascading failures. Model degradation involves configuring a fallback strategy for the model at the API gateway layer, allowing requests to be forwarded directly to the cloud model when a self-deployed model call fails.

9. The degradation and disaster recovery system for a large language model intelligent agent application according to claim 8, characterized in that: The strategy engine layer also provides proactive responses for degradation / disaster recovery strategies, including: a. For model unavailability: The triggering condition is N consecutive health check failures or the error rate / timeout threshold being exceeded, or a model-level DOWN alarm; the actions to be performed include: routing all requests within the affected area to the cloud pool; marking the model as maintenance mode and writing it to the event database; continuously monitoring the model's recovery status; if recovery fails, triggering an alarm to notify operations and maintenance; the recovery logic is that after the monitoring layer reports UP, wait for a stable window, and then return traffic to the self-deployed system according to a phased rollback strategy; b. Regarding model performance degradation: The trigger condition is that the P95 index of TTFT / RT exceeds the threshold, and the threshold is set according to different models; the execution actions include calling the gateway POST / gateway / flowcontrol / model to update the model routing weights; the back-switch logic is a phased back-switch, and the stability and error rate are verified at each step; c. For workflow orchestration layer anomalies: The triggering conditions are an increase in the error rate of the primary instance, excessively long task queues, or failed health checks; The action is to switch the request to the backup instance by calling the application service layer's POST / app / workflowcontrol / {wokflowId} interface through the strategy engine; The rollback logic is a phased rollback, with stability and error rate verified at each step. d. Assurance for business metrics driven by performance indicators: The trigger condition is that the SLI of critical business is lower than the SLO; the actions include calling GET / app / flowcontrol / rules to obtain the application priority configuration, reducing or pausing low-priority applications, and adjusting the application quota by calling POST / app / flowcontrol / {appId} to close the quota of low-priority applications; the rollback logic is a phased rollback, with the quota of low-priority applications decreasing by 10% in each step, and the stability of high-priority applications is verified at each step.

10. The degradation and disaster recovery system for a large language model intelligent agent application according to claim 9, characterized in that: The strategy engine layer also includes strategy priority, conflict resolution, and idempotency, wherein... The strategy priority mechanism assigns a priority field to each strategy. When an event is triggered, the strategy engine evaluates the applicable strategies in descending order of priority. When a mutually exclusive strategy is encountered, a conflict resolution strategy is used.

Images