A method, system, and computer-readable storage medium for on-demand scaling down and automatic recovery of AI agents.

Abstract

This application relates to the field of artificial intelligence service deployment and container orchestration technology, and discloses a method, system, and computer-readable storage medium for on-demand scaling and automatic recovery of AI Agents. The method is applied to a container orchestration platform deploying AI Agent workloads, an external scheduling controller, an intelligent network agent, and persistent storage. When the AI ​​Agent workload is detected to meet idle conditions, the replica count is adjusted to 0. After the AI ​​Agent instance terminates, the intelligent network agent maintains a remote connection with the remote message service and receives messages. When a scheduled task trigger event, an external request trigger event, or a message arrival trigger event is detected, the replica count is adjusted to a non-zero value to restore instance operation. After instance recovery, the business state is loaded from persistent storage, and cached messages and / or corresponding business processes are processed. This solution can reduce resource consumption during idle periods while ensuring business continuity.

Description

Technical Field

[0001] This invention relates to the field of containerized intelligent agent operation management technology, specifically to a method, system, and computer-readable storage medium for on-demand scaling and automatic recovery of AI agents. Background Technology

[0002] With the development of general artificial intelligence capabilities, an increasing number of AI agents are being deployed as long-running service instances to handle instant messaging, execute scheduled tasks, respond to external callbacks, and invoke external toolchains. These AI agents are typically not offline programs that execute only once, but rather online workloads that need to run continuously and interact with external systems. To achieve better deployment flexibility, isolation, and observability, the industry commonly adopts containerization to host these AI agents, further deploying them on container orchestration platforms such as Kubernetes. This enables unified operation and maintenance through capabilities such as workload replica management, service discovery, health checks, and persistent volume mounting. In scenarios such as instant messaging bots, collaboration assistants, automated operation and maintenance agents, and workflow execution agents, this deployment method has gradually become an important implementation path for general-purpose AI agents, enabling these agents to continuously provide capabilities as online services by leveraging the scheduling, elastic scaling, operational isolation, and state-bearing capabilities of container orchestration platforms.

[0003] Unlike traditional stateless microservices, general-purpose AI agents typically undertake multiple technical responsibilities simultaneously during operation. First, AI agents need to connect to external messaging channels, such as enterprise collaboration platforms, instant messaging platforms, or open messaging platforms, to receive user input, event notifications, or platform pushes. To achieve continuous access, these AI agents usually establish long-lived connections such as WebSockets with remote messaging services and periodically perform connection keep-alive or heartbeat interactions. Second, AI agents often need to execute scheduled tasks, such as scheduled reminders, scheduled inspections, scheduled synchronization, scheduled analysis, and scheduled summarization. Therefore, they are often configured with cron-like scheduling logic or other time-triggered logic. Third, as AI agents continuously process dialogues, tasks, and tool calls, they also gradually form business states such as session context, authentication credentials, task queues, plugin status, and channel configurations. These states affect subsequent message processing and task execution results. Therefore, an AI agent instance is not merely a simple request processing process during runtime; it is often the message receiving entity, the scheduling and execution entity, and the entity that carries the business context.

[0004] To ensure the continued effectiveness of these capabilities, a common practice in existing systems is to keep AI Agent instances running for extended periods. Even when there are no user messages, external webhook requests, or pending tasks, the system tends to keep at least one instance online to prevent message channels from being interrupted, scheduled tasks from stopping, or business contexts from becoming invalid. While this persistent deployment approach helps reduce the probability of operational interruptions, it also leads to higher resource consumption. For a single Agent, CPU and memory usage during idle periods may seem limited; however, in large-scale SaaS platforms, multi-tenant private platforms, or environments that need to simultaneously support a large number of Agent instances, the long-term persistence of numerous idle instances leads to low overall resource utilization, thereby increasing infrastructure costs.

[0005] To achieve the goal of "releasing resources during idle periods and resuming operation when business demands arise," existing container platforms and the serverless domain have developed several related technologies. For example, Kubernetes natively supports adjusting the number of workload replicas, reducing the number of instances when applications are not under load. Furthermore, serverless frameworks like Knative build scale-to-zero capabilities on top of Kubernetes, allowing applications to shrink to zero running replicas when there are no requests and restart instances when new requests arrive. For such solutions, the platform typically receives HTTP requests or request metrics from externally resident components and then triggers instance scaling accordingly, forming an execution chain of "request arrival—instance recovery—request forwarding." Meanwhile, event-driven scaling solutions like KEDA also support triggering workload scaling based on cron expressions, external metrics, queue lengths, or other event sources. Therefore, existing technologies can already support the basic pattern of "scaling down workloads to zero replicas and then restoring them" to a certain extent.

[0006] However, the aforementioned existing technologies primarily target stateless, short-connection, request-response general services, and their technical premises are not entirely consistent with the AI ​​Agent scenario. For general HTTP services, requests typically arrive discretely, and service instances can be restored by a middleware layer after a request arrives, processing the request only after the instance has recovered. The request itself does not assume a continuous online state; therefore, as long as the middleware layer can handle the request within a short period, the backend instance can continue processing business logic after restarting. In contrast, AI Agents in many scenarios do not simply wait for discrete HTTP requests but need to continuously receive push notifications from remote messaging platforms and maintain an online state and connection session. When an AI Agent instance directly handles this long connection, the connection will immediately break if the instance stops. A broken connection not only means that the local machine cannot continue receiving messages but may also cause the remote messaging service to consider the Agent offline, thus affecting subsequent push notifications and session persistence. Therefore, the "restore after request buffering" approach in general serverless solutions is difficult to directly address the long-connection access problem in AI Agents.

[0007] Furthermore, while existing platform solutions for AI Agents have begun to address the issue of agent runtime resources, their optimization directions may not entirely overlap with the problems in this field. Some solutions reduce the cost of agent persistence by rewriting the runtime, simplifying the framework, or reducing the memory overhead of a single instance; some solutions support instance reconstruction through external state or workflow checkpoint mechanisms; and still others model the agent as a stateful workload with persistent data to restore the environment after the instance restarts. These solutions are effective in reducing unit instance costs, improving deployment efficiency, or enhancing state recovery. However, if their connectivity remains bound to the instance, or their scheduling capabilities still depend on the instance's continuous operation, remote message access and time-triggered capabilities may still be interrupted after the instance stops. In other words, while existing technologies focus on resource optimization, state recovery, or event-driven approaches, they rarely treat the overall relationship between "scheduling capabilities, connectivity capabilities, and business state" and the instance lifecycle in AI Agent scenarios as a unified problem.

[0008] Further analysis from the perspective of scaling technology reveals that existing scaling capabilities based on external events typically focus more on "when to scale" and "how many replicas to scale up to," while paying less attention to who will replace the original instance to continue handling the message channel after the instance replica count is reduced to zero, who will be responsible for detecting subsequent scheduling conditions, and how to resume the business processes before the interruption after the instance recovers. For example, KEDA's event-driven mechanism can drive workload recovery based on cron schedules, external metrics, or message queue status, but it is not responsible for maintaining the connection to the remote messaging platform, nor for caching business messages when the instance is offline and delivering them in an orderly manner after the instance recovers. Similarly, Knative's Activator and other components can temporarily handle request traffic when HTTP requests arrive, but this capability is mainly designed for short HTTP connection traffic and is not naturally suitable for AI Agent applications that require continuous remote connection status. For AI Agents, if there is no independent component to continue handling message access during instance downtime, even if the platform has scale-to-zero capabilities, it is difficult to guarantee that message events can reliably trigger subsequent recovery processes.

[0009] On the other hand, AI Agent instance recovery differs from the startup of ordinary stateless services. Stateless HTTP services, once started, can typically receive requests as long as the process is available and passes health checks. However, AI Agent recovery often requires loading a series of business continuity-related states. For example, the main configuration file determines the instance's identity and access channel; authentication information determines whether it can re-establish necessary access capabilities; session state affects whether it can continue dialogue using existing context; the task queue affects whether it will miss tasks generated but not yet completed before shutdown; and plugin configuration and plugin context affect subsequent tool call paths. If the loading, association, and recovery order of these states lack appropriate design, even if the instance is recreated, it may only be "started at the process level" and not truly reach the state of "business-level continued processing." This makes the recovery problem in AI Agent scenarios often not just a matter of whether the container has been successfully started, but also involves when the instance can receive messages again, when it can resume executing tasks, and when it can be considered truly recovered.

[0010] Some existing solutions for stateful workloads recognize the importance of state persistence and restore data after instance reconstruction through persistent volumes, object storage, or state services. These solutions offer valuable insights for restoring state in file system environments, development environments, or general workflows. However, for general AI agents, the states that truly impact business continuity are not limited to file system content; they also include business-level states such as session-level context, authentication tokens, scheduling configurations, task queues, and plugin-level runtime states. Furthermore, the restoration of these states needs to be coordinated with message access timing, instance readiness assessment, and subsequent event forwarding processes. If state restoration is disconnected from message continuation, problems can easily arise such as instance endpoints being visible externally but their internal states not yet fully restored, or cached messages being forwarded prematurely before the instance has a complete business context. Therefore, even if existing technologies possess some degree of "state recoverability," they may not naturally meet the business continuity requirements of AI agent scenarios.

[0011] In summary, at least three interrelated technical contradictions exist in the deployment scenario of general AI Agents for container orchestration platforms. First, AI Agents often need to run continuously to maintain long-term connection access and scheduled task execution, but this continuous operation leads to prolonged resource occupation during idle periods. Second, while container platforms support scaling down to zero replicas and restoring instances upon events, existing scaling-down recovery solutions primarily target short-connection HTTP services, rarely addressing the issues of maintaining long connections and resuming message transmission after instance shutdown. Third, although AI Agents can retain some runtime state through persistence, without coordination with recovery triggering, connection access, and message forwarding mechanisms, it is still difficult to guarantee business continuity after recovery. In other words, in the specific application scenario of AI Agents, existing technologies often only solve one aspect of the problems of "scaling down," "triggering recovery," or "state preservation," making it difficult to simultaneously address all three aspects: releasing resources when instances are idle, maintaining necessary triggering capabilities during instance offline periods, and resuming original business processes after instance recovery.

[0012] Therefore, for general-purpose AI agents deployed in container orchestration platforms, how to reduce the workload to zero running replicas when the instance is idle to free up computing and memory resources, while reliably triggering instance recovery when scheduled tasks expire, external webhook requests arrive, or messages arrive from remote messaging platforms, and continuing to process the original business context after instance recovery, remains a problem that needs further improvement in existing technologies. More specifically, existing technologies in AIAgent scenarios still pay relatively little attention to the following issues: how to maintain necessary external trigger detection capabilities when the instance is not running; how to continue accepting remote message access when the instance is not running; and how to continue message and task processing after the instance is recreated and its state is restored. The existence of these problems makes it difficult for containerized general-purpose AI agents to balance resource utilization and business continuity, thus necessitating the development of new technical solutions for improvement. Summary of the Invention

[0013] This application relates to the field of artificial intelligence service deployment and container orchestration technology, and in particular to a method, system and computer-readable storage medium for on-demand scaling and automatic recovery applicable to AIAgent.

[0014] Containerized AI agents typically need to run for extended periods to handle instant messages, execute scheduled tasks, respond to external requests, and maintain connections with remote messaging services. In existing solutions, the relevant trigger detection capabilities and message channel maintenance capabilities usually rely on the AI ​​agent instance itself remaining continuously online. Once the workload is scaled down to zero replicas, issues often arise such as scheduled tasks failing to trigger, external requests failing to wake up the system, remote connections being interrupted, and business status being difficult to maintain. This makes it difficult for AI agents to strike a balance between resource conservation and maintaining business continuity.

[0015] One object of this application is to provide an on-demand scaling-up and automatic recovery method, system, and computer-readable storage medium for AI Agents, so that the AI ​​Agent workload can scale down to zero copies when idle conditions are met, and automatically resume operation when a triggering event is detected, while continuing to process cached messages or triggering business after recovery.

[0016] To achieve the above objectives, in one aspect of this application, an on-demand scaling-up and automatic recovery method for AI Agents is provided, applied to a container orchestration platform. The container orchestration platform deploys an AI Agent workload, an external scheduler controller, an intelligent network agent, and persistent storage. The method includes: when the external scheduler controller detects that the AI ​​Agent workload meets an idle condition, it calls the workload control interface of the container orchestration platform to adjust the replica count of the AI ​​Agent workload to 0, thereby terminating the operation of the AI ​​Agent instance; after the AI ​​Agent instance terminates, the intelligent network agent maintains a remote connection with a remote message service and receives messages sent by the remote message service; the external scheduler controller detects a scheduled task trigger event or an external request trigger event, and / or the intelligent network agent detects a message arrival trigger event, and upon detecting any trigger event, calls the workload control interface to adjust the replica count of the AI ​​Agent workload to a non-zero value, thereby restoring the operation of the AI ​​Agent instance; after the AI ​​Agent instance resumes operation, the AI ​​Agent... The Agent instance loads the business state from the persistent storage and processes the cached message corresponding to the message arrival trigger event and / or the business corresponding to the scheduled task trigger event or the external request trigger event.

[0017] In one aspect of this application, an on-demand scaling-up and automatic recovery system for AI Agents is also provided, applied to a container orchestration platform, comprising: an AI Agent workload for running AI Agent instances; an external scheduling controller for, when detecting that the AI ​​Agent workload meets an idle condition, calling the workload control interface of the container orchestration platform to adjust the number of replicas of the AI ​​Agent workload to 0, and when detecting a scheduled task trigger event or an external request trigger event, calling the workload control interface to adjust the number of replicas of the AI ​​Agent workload to a non-zero value; an intelligent network proxy for, maintaining a remote connection with a remote message service after the AI ​​Agent instance terminates operation, receiving messages sent by the remote message service, and calling the workload control interface to adjust the number of replicas of the AI ​​Agent workload to a non-zero value when detecting a message arrival trigger event; and persistent storage for storing the business state required for the AI ​​Agent instance to resume operation; wherein, after the AI ​​Agent instance resumes operation, it loads the business state from the persistent storage and processes cached messages and / or the business corresponding to the scheduled task trigger event or the external request trigger event.

[0018] In one aspect of this application, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the above-described method.

[0019] After adopting the above technical solution, the AI ​​Agent workload can be scaled down to zero replicas when idle conditions are met. Even after the AI ​​Agent instance terminates, the remote message channel can still be maintained by the intelligent network agent, and the external scheduling controller can continue to detect scheduled task trigger events or external request trigger events, thereby restoring the AI ​​Agent instance to operation when a trigger event is detected. The restored AI Agent instance can also load business state from persistent storage and continue to process cached messages or trigger business, thereby reducing resource consumption during idle periods and ensuring the continuity of AI Agent business. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments in this specification, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a general system architecture diagram of one embodiment of this application.

[0022] Figure 2 This is a flowchart illustrating the overall method of one embodiment of this application.

[0023] Figure 3 This is a flowchart illustrating a shrinkage closed-loop process according to an embodiment of this application.

[0024] Figure 4 This is a flowchart illustrating the recovery closed-loop process of one embodiment of this application.

[0025] Figure 5 This is a functional structure diagram of an external scheduling controller according to an embodiment of this application.

[0026] Figure 6 This is a functional structure diagram of an intelligent network agent according to an embodiment of this application.

[0027] Figure 7 This is a connection relationship state transition diagram according to an embodiment of this application.

[0028] Figure 8 This is a timing diagram of message arrival triggering recovery according to one embodiment of this application.

[0029] Figure 9 This is a timing diagram of a scheduled task triggering and an external request triggering recovery according to an embodiment of this application.

[0030] Figure 10 This is a schematic diagram of the business state storage content according to one embodiment of this application.

[0031] Figure 11 This is a flowchart illustrating the state recovery and serviceability determination process according to one embodiment of this application.

[0032] Figure 12 This is a system deployment implementation diagram of one embodiment of this application. Detailed Implementation

[0033] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0034] Figure 1 The diagram illustrates the overall system architecture. The container orchestration platform 100 can deploy an AI Agent workload 110, an external scheduling controller 120, an intelligent network agent 130, and persistent storage 140. The AI ​​Agent workload 110 is used to create and manage AI Agent instances 111. The external scheduling controller 120 is deployed separately from the AI ​​Agent instances 111, enabling it to continue performing detection and control operations when the AI ​​Agent instances 111 are not running. The intelligent network agent 130 is also deployed separately from the AI ​​Agent instances 111, maintaining a connection with the remote messaging service 150 and taking over message receiving and caching operations during the shutdown of the AI ​​Agent instances 111. The persistent storage 140 is used to store the business state required for the AI ​​Agent instances 111 to resume operation. The workload control interface 160 interacts with the container orchestration platform 100 to adjust the number of replicas of the AI ​​Agent workload 110.

[0035] The operational basis of this embodiment is not to keep the AI ​​Agent instance 111 continuously online, but to place the capabilities that still need to be retained after the instance is stopped outside the instance. The external scheduling controller 120 retains the trigger detection capability, the intelligent network agent 130 retains the message channel maintenance capability, and the persistent storage 140 retains the business state carrying capability. After this processing, the AI ​​Agent instance 111 can exit operation when idle, while the three conditions directly related to subsequent recovery—detection, connection, and state—still exist. What is formed is not a simple stop and start relationship, but a continuously connectable operational chain.

[0036] Figure 2 This shows the overall methodology and process. Figure 2 The idle detection 201, scaling down to 0202, remote connection maintenance 203, event trigger detection 204, instance recovery 205, state loading 206, and business processing 207 constitute a complete main process. This main process can be mapped to... Figure 1 The various participating entities are as follows: Idle detection 201 is mainly performed by the external scheduling controller 120; scaling down to 0202 is applied to the AI ​​Agent workload 110 through the workload control interface 160; remote connection maintenance 203 is mainly performed by the intelligent network agent 130; trigger event detection 204 is handled by the external scheduling controller 120 and the intelligent network agent 130 respectively; instance recovery 205 is performed by the container orchestration platform 100 to recreate the AI ​​Agent instance 111 according to the change in the number of replicas; state loading 206 is performed by the recovered AI Agent instance 111 to read the business state from the persistent storage 140; and business processing 207 is performed by the AI ​​Agent instance 111 to process cached messages or triggered business.

[0037] In one embodiment of this application, the AI ​​Agent workload 110 can be implemented using a deployment object approach. For example, a Deployment can be used for replica management, or a StatefulSet can be used for workload management with stable identifiers. If the business focus is on creating and destroying running instances on demand, and the business state is mainly stored externally through persistent storage 140, then using a Deployment will make it easier to utilize mature replica count control mechanisms. If the business also requires more stable network identifiers or dedicated volume mappings, a StatefulSet can also be used. In both of these implementation methods, the AI ​​Agent instance 111 can be controlled to 0 or restored to a non-zero value through replica count control, thus not affecting the overall principle of this embodiment.

[0038] Figure 5The diagram illustrates one implementation of the external scheduling controller 120. The scheduling detection module 121 can read the scheduled task configuration 143 and determine the next task trigger time accordingly. The trigger detection module 122 can receive external requests or detect the task queue 145, event flag files, and other pending items. The idle determination module 123 can summarize the existence of pending tasks, pending external trigger requests, pending forwarded cached messages, and scheduled tasks that are about to expire. The workload control module 124 then calls the workload control interface 160 based on the idle determination result or the trigger detection result. This control interface can be represented as a replica number adjustment interface provided by the container orchestration platform 100, or as a control service interface encapsulated by the platform gateway. The choice of interface form is not crucial; the key is that the interface enables the AIAgent workload 110 to switch between zero replicas and non-zero replicas.

[0039] Figure 6 The diagram illustrates one implementation structure of the intelligent network agent 130. The remote connection maintenance module 131 is responsible for maintaining a remote connection 172 with the remote message service 150. The message receiving module 132 is responsible for receiving messages from the remote message service 150. The message caching module 133 is responsible for temporarily storing messages when the AI ​​Agent instance 111 is not running or has not yet reached a serviceable state. The recovery triggering module 134 is responsible for triggering the AI ​​Agent workload 110 to resume operation when a message arrives and the AI ​​Agent instance 111 is unavailable. The message forwarding module 135 is responsible for forwarding cached messages or subsequent real-time messages to the AI ​​Agent instance 111 after it recovers. This division of labor ensures that the intelligent network agent 130 is not simply a transparent forwarding node, but an intermediary entity capable of assuming message channel responsibilities during instance lifecycle changes.

[0040] Figure 7This illustrates the state transition of the connection relationship. A local connection 171 can be established between AI Agent instance 111 and intelligent network agent 130, and a remote connection 172 can be established between intelligent network agent 130 and remote message service 150. When AI Agent instance 111 is running normally, both local connection 171 and remote connection 172 are connected. Messages can enter AI Agent instance 111 via intelligent network agent 130, or in some implementations, the agent can perform protocol adaptation before forwarding. After AI Agent instance 111 is scaled down, local connection 171 is interrupted, but remote connection 172 is still maintained by intelligent network agent 130. This is because the remote message service 150 is no longer aware of the connection as the AI ​​Agent instance 111, which has exited operation, but rather of the continuously existing intelligent network agent 130. After AI Agent instance 111 resumes operation, local connection 171 can be re-established, and cached messages can then be sent to AI Agent instance 111 via this connection. Figure 7 The connection state changes in the process do not require the local connection 171 to adopt a specific protocol. The local connection 171 can be implemented using local sockets, Hypertext Transfer Protocol (HTTP), General Remote Procedure Call Protocol (gRPC), or WebSocket, as long as it can provide a reliable local interaction path after the AIAgent instance 111 recovers.

[0041] Figure 10 This diagram illustrates the business states stored in persistent storage 140. The main configuration file 141 describes the basic identity, access channel, plugin activation information, and operating parameters of the AI ​​Agent instance 111. Authentication credentials 142 describes the authentication information required to access the remote messaging service 150 or other third-party systems. Scheduled task configuration 143 describes scheduled tasks and their triggering rules. Session data 144 describes the dialogue context, session index, or other states related to continuous interaction. Task queue 145 describes tasks to be executed, tasks in execution, or results awaiting confirmation. Plugin status 146 describes plugin loading information, plugin context, or other auxiliary operating states. After these states are placed in persistent storage 140, the continued existence of the AI ​​Agent instance 111 no longer determines whether the business state continues. With the state bearer and instance bearer separated, the AI ​​Agent instance 111 can only re-establish its operating context externally upon recovery.

[0042] Business states can be stored in various formats. The main configuration file 141 and scheduled task configuration 143 can be saved as files; authentication credentials 142 can be persisted via key storage or platform encrypted services; session data 144 and task queues 145 can be stored in a database, key-value store, or a local persistent volume with logs; and plugin states 146 can be stored in structured files or object storage. Using different storage media for different types of states does not affect the overall structure, as long as the AI ​​Agent instance 111 can read these states through a unified access path during runtime recovery. In one implementation example, persistent storage 140 can be a hybrid of persistent volumes and a database. This approach is based on the consideration that configuration and file-based data are suitable for volume mounting and recovery, while queue and session-based data are more suitable for random access or sequential consumption.

[0043] Figure 12 This illustrates one deployment implementation. The external scheduling controller Pod 1210, the intelligent network agent Pod 1310, and the AI ​​Agent Pod 1110 can be deployed on different nodes within the container orchestration platform 100. The persistent volume 1410 can be mounted to the AI ​​Agent Pod 1110, or it can be accessed via a shared storage path for the external scheduling controller 120 to read the scheduled task configuration 143 or for the intelligent network agent 130 to read necessary authentication information. The service 1610 can provide a discoverable service entry point for the AI ​​Agent Pod 1110, and can also serve as an access point for the external scheduling controller 120 or the intelligent network agent 130 to access the workload control interface 160. In some deployment environments, the external scheduling controller 120 and the intelligent network agent 130 can be designed as platform-level common components, providing services to multiple AI Agent workloads 110; in other deployment environments, the external scheduling controller 120 and the intelligent network agent 130 can be deployed separately for each AI Agent workload 110. The main differences between the two methods lie in the isolation strength, resource usage, and operational complexity, but do not affect the basic implementation of scaling down and recovery links in this embodiment.

[0044] Figure 3 and Figure 4 The closed-loop relationship in can be found in Figure 1 This is implemented within the framework. Figure 3 The corresponding shrinkage loop focuses on when the AI ​​Agent workload 110 can be moved from the running state to the zero-replica state. Figure 4The corresponding recovery loop focuses on which type of triggering event can bring the system back to running state after AI Agent instance 111 stops. Considering these two loops separately helps avoid mixing downsizing and recovery conditions. Downsizing conditions are biased towards passive judgment, meaning there are currently no events requiring AI Agent instance 111 to continue running; recovery conditions are biased towards active judgment, meaning events have occurred that require AI Agent instance 111 to resume processing. The external scheduling controller 120 and the intelligent network agent 130 each handle different parts of these two judgments, therefore the system will not lose control due to AI Agent instance 111 exiting operation.

[0045] In some implementations, the idle condition is not simply equivalent to a no-message state. If the lack of new messages is used solely as the basis for scaling down, the AI ​​Agent instance 111 may be prematurely terminated when there are still pending tasks or when scheduled tasks are about to expire. The idle determination module 123 typically needs to consider whether the task queue 145 is empty, whether the external request cache is empty, whether the cache queue of the intelligent network agent 130 is empty, and whether the next trigger time indicated by the scheduled task configuration 143 is far from the current time. The determination here can be a logical AND relationship, rather than a logical OR relationship. With this determination structure, a more complete safety boundary can be maintained before the AI ​​Agent workload 110 is scaled down to zero replicas.

[0046] In one embodiment, the external scheduling controller 120 can calculate whether there are any scheduled tasks that are about to expire based on the trigger time of the next scheduled task. If the current time is denoted as... The next scheduled task trigger time is recorded as... The preset advance window is denoted as Then the remaining waiting time of the scheduled task can be calculated using the following formula. : (1) when When this happens, it can be determined that there are scheduled tasks that are about to expire; when When this happens, it can be determined that there are no scheduled tasks expiring within the advance window. In equation (1) Indicates the current detection time. This indicates the next task trigger time, obtained from parsing the scheduled task configuration 143. This represents a preset time threshold. Based on this calculation result, the idle time determination module 123 can incorporate the situation where a timed task is about to be triggered into the scaling-down determination process, instead of waiting until the task actually reaches its trigger point. This reduces the oscillation phenomenon of immediate recovery after scaling down.

[0047] From the perspective of the overall operation process, Figures 1 to 12 The structure shown is not a parallel stack, but rather contains a clear chain of constraints. Whether the AI ​​Agent workload 110 runs is controlled by the workload control interface 160; when the workload control interface 160 acts is affected by the detection results of the external scheduling controller 120 and the intelligent network agent 130; whether the AI ​​Agent instance 111 can continue to process historical business after recovery is affected by whether the business status in the persistent storage 140 is complete; when the intelligent network agent 130 forwards cached messages is affected by whether the AI ​​Agent instance 111 has reached a serviceable state. It is precisely because of the above-mentioned coupling relationships between the participating entities in terms of structure and timing that the system can maintain business continuity without relying on the AI ​​Agent instance 111 to run resident.

[0048] Depend on Figures 1 to 12 The resulting embodiment yields the following technical results: The AI ​​Agent workload 110 can be adjusted to zero replicas in an idle state, and the AI ​​Agent instance 111 does not need to continuously occupy computing resources; after the AI ​​Agent instance 111 terminates, the remote connection 172 can still be maintained by the intelligent network agent 130, thus the message access link will not disappear entirely due to instance exit; the external scheduling controller 120 and persistent storage 140 ensure that timed triggering, external request triggering, and business state recovery can continue, and the AI ​​Agent instance 111 can continue processing cached messages or other triggered services after recreation. This structural arrangement ensures that resource release and business continuity are no longer mutually exclusive.

[0049] Figure 3 This illustrates a process relationship of a shrinking closed loop. The AI ​​Agent workload 110 enters the zero-replica state from the running state not solely based on whether the current instantaneous load is zero, but rather based on the result obtained by the external scheduling controller 120 after combining and judging multiple operating conditions. Figure 3 The idle condition detection 301, continuous detection 302, scaling down determination 303, replica count adjustment to 0 304, instance termination 305, and agent maintaining remote connection 306 constitute a scaling down path suitable for engineering implementation. The characteristic of this path is that the scaling down action does not rely on the AI ​​Agent instance 111 itself to decide, but is judged and controlled externally by the external scheduling controller 120. After this processing, even if the AI ​​Agent instance 111 has exited operation, the system still retains the detection components required for subsequent recovery.

[0050] In one embodiment, the external scheduling controller 120 can periodically check whether the AI ​​Agent workload 110 meets the idle condition. The idle condition can be composed of at least four states: one is the pending task state, another is the pending external trigger request state, another is the pending forwarded cached message state, and another is the soon-to-expire scheduled task state. If there are still pending tasks in the task queue 145, it means that the task processing will be interrupted after the AI ​​Agent instance 111 exits; if the external scheduling controller 120 has received an external request that has not yet been forwarded, it means that the instance still needs to be retained or restored as soon as possible; if there are still pending messages in the cache queue of the intelligent network agent 130, it means that there is already pending content on the remote message access link; if the next trigger time corresponding to the scheduled task configuration 143 is close to the current time, it is not suitable to immediately scale down. Only when these states collectively indicate that there is no need to continue running in the short term will the external scheduling controller 120 enter the scaling down determination phase.

[0051] The judgment here is closer to a combination constraint than a single point trigger. In actual deployment, if only one state judgment is used, it is easy for the number of replicas to fluctuate too frequently. For example, if only whether the task queue 145 is empty is used as the criterion for scaling down, the AI ​​Agent instance 111 may be prematurely terminated when there are no tasks to be executed but there are still scheduled tasks that are about to expire; if only whether the message cache is empty is used as the criterion, the AI ​​Agent instance 111 may be prematurely scaled down in a window when messages are temporarily quiet but external requests may still arrive at any time. Therefore, the external scheduling controller 120 usually does not use single-condition judgment, but will summarize all state variables into the idle judgment module 123 to obtain a more stable scaling down judgment result.

[0052] In one implementation example, the idle determination module 123 can generate a corresponding determination quantity for each type of state. The number of tasks to be processed is denoted as... The number of pending external requests is denoted as The number of cached messages to be forwarded is denoted as The timed task is nearing its trigger mark as . When there are no scheduled tasks that are about to expire, When there are scheduled tasks that are about to expire, Based on this, an idle determination quantity can be constructed. : (2) In equation (2) Represents the logical AND operation. When When, it indicates that the AI ​​Agent workload 110 meets the idle condition; when When there is a factor that is not suitable for scaling down, it means that there is at least one factor that is not suitable for scaling down. The advantage of this definition is that the external scheduling controller 120 can use a unified decision quantity to accept the detection results from different sources, which is convenient for subsequent continuous detection and scaling down execution. Equation (2) itself is not the only implementation method. It can also be replaced by a Boolean state table, rule engine or other logical expression, as long as the combination judgment relationship can be reflected in the end.

[0053] For scheduled tasks that are about to expire, the external scheduling controller 120 can calculate the next trigger time based on the trigger rules in the scheduled task configuration 143. If the current detection time is... The next scheduled task will be triggered at [time]. The default advance window is Then when At this point, the scheduled task can be considered to have entered the near-triggering range. In engineering implementation, The settings can be configured based on the average recovery time, state loading time, and message channel establishment time of AI Agent instance 111. If... If the size is too small, AI Agent instance 111 may be scaled down and then immediately restored due to the expiration of the scheduled task; if... An excessively high value could cause AI Agent instance 111 to remain running for extended periods of idle time. This parameter is typically configured by operations and maintenance personnel or preset by platform policies and is not required to be a fixed value in all deployment environments.

[0054] Figure 3 The continuous detection 302 in the code corresponds to a commonly used anti-shake processing method. Even if the idle determination module 123 obtains an idle state during a certain detection... The workload control module 124 may not immediately issue a scaling-down command. This is because task status, request status, and message status can all fluctuate in the short term. Especially in message-based scenarios, the AI ​​Agent instance 111 may suddenly become quiet for tens of seconds, only to receive new messages shortly thereafter. If the idle status is responded to immediately, the AI ​​Agent workload 110 is prone to frequent switching between running, scaling down, and recovering within a short period. Such fluctuations not only increase the scheduling burden on the container orchestration platform 100 but also lengthen the processing latency of actual business operations.

[0055] To mitigate the aforementioned problems, in one embodiment, the external scheduling controller 120 can perform checks according to a preset cycle. Perform repeated detection, and in succession The scaling-down operation will only be performed if all tests meet the idle condition. If the first test... The test result is recorded as follows: The continuous detection condition can be expressed by the following formula: (3) In equation (3) This represents a series multiplication operation. Because... Only take 0 or 1, when the most recent consecutive When all tests meet the idle condition If any one of the checks fails to meet the idle condition, In this implementation, Indicates the preset inspection cycle. This indicates the number of consecutive checks. With this processing, the scaling down action is no longer triggered directly by a single instantaneous check, but rather by a stable idle state within a specified observation window, thus reducing operational jitter.

[0056] In addition to continuous detection conditions, the scaling-down closed loop can also introduce runtime cooling constraints. After AI Agent instance 111 recovers to a serviceable state, it typically experiences a short period of message fallback or task processing completion. If it immediately re-enters idle detection and scales down during this phase, the instance may quickly exit again after recovery. In one embodiment, the external scheduling controller 120 can record the recovery time after AI Agent instance 111 has recovered. And set the minimum duration. At the current moment satisfy At that time, the idle determination module 123 can detect it in time. Scaling down will not be performed for the time being. After this processing, AI Agent instance 111 can still run for an observation period after recovery to absorb subsequent message or request fluctuations.

[0057] Once the workload control module 124 determines that scaling down is allowed, it can send a replica count adjustment request to the container orchestration platform 100 via the workload control interface 160. Figure 3 The setting of the replica count to 0304 indicates that the target replica count for AI Agent workload 110 is set to 0. This adjustment can be applied directly to the replica count attribute of a Deployment or StatefulSet, or it can be accomplished through the platform's management interface. After this adjustment is executed by the container orchestration platform 100, AI Agent workload 110 will no longer maintain a running instance, and AI Agent instance 111 will subsequently enter the termination process. Figure 3 The instance in the code terminated with error 305.

[0058] An instance termination error 305 does not mean that all related connections and states disappear at the same time. Figure 7The connection state changes have been described. After AI Agent instance 111 terminates, the local connection 171 between it and intelligent network agent 130 can be interrupted. This interruption stems from the instance's own exit and does not require intelligent network agent 130 to actively simulate the instance maintaining the local connection. In contrast, the remote connection 172 between intelligent network agent 130 and remote messaging service 150 remains in a persistent state after scaling down. Figure 3 The agent in the process continues to maintain a remote connection 306. This state arrangement has a direct consequence: the exit of AI Agent instance 111 will not directly cause the entire remote message channel to fail. The scaling-down operation affects local processing capacity, not remote message access capacity.

[0059] In some implementations, the intelligent network agent 130 not only maintains the remote connection 172 after the AI ​​Agent instance 111 exits operation, but also maintains the keep-alive interactions required for the remote connection 172. For example, if the remote messaging service 150 uses the WebSocket protocol, the intelligent network agent 130 can send heartbeat frames, respond to Ping, or maintain the validity of the session token as required by the protocol. This behavior falls within the functional scope of the remote connection maintenance module 131. The purpose of this design is that the remote messaging service 150 becomes aware of the online session through the intelligent network agent 130, rather than the AI ​​Agent instance 111 that has exited operation. Therefore, whether subsequent messages can continue to be delivered depends on whether the intelligent network agent 130 remains online, not on whether the AI ​​Agent instance 111 is running.

[0060] The scaling-down closed-loop process involves a trade-off that is easily overlooked during engineering deployment. If the idle condition is too strict, the AIAgent workload 110 will have difficulty entering a zero-replica state, weakening the resource release effect; if the idle condition is too lenient, although scaling down is easier, the AI ​​Agent instance 111 may terminate prematurely when there is still potential business demand, leading to frequent subsequent recovery. This trade-off is usually not resolved by changing the overall architecture, but rather by adjusting the threshold, window, and number of consecutive checks in the idle determination module 123. For example, for high-frequency message scenarios, the minimum retention time can be appropriately extended. And increase the number of consecutive tests. For low-frequency, long-idle scenarios, the inspection cycle can be appropriately shortened. And reduce the advance window This allows the AI ​​Agent workload 110 to more aggressively enter a zero-replica state. These parameter adjustments do not change... Figure 3 The shrinking closed-loop logic structure shown only affects the trigger sensitivity of the structure under different business loads.

[0061] In some deployment environments, the external scheduling controller 120 may not necessarily read all the original states independently. One feasible approach is to have the task queue 145 state provided by the task storage service, the session state by the session state service, the cached message count by the intelligent network agent 130, and the scheduled task trigger time parsed by the scheduled task configuration 143. The idle time determination module 123 can then summarize these results. This approach avoids centralizing the reading processes of different business logics into a single component and facilitates platform-based deployment. In another implementation, the external scheduling controller 120 can also directly access a shared database or shared storage to uniformly read the aforementioned states. The main difference between the two approaches lies in deployment complexity and the degree of decoupling, without affecting the scaling and closed-loop mechanism itself.

[0062] Figure 3 The closed loop implies that the scaling down and recovery actions are not two independent, isolated events. Before scaling down, the external scheduling controller 120 has already reserved the necessary conditions for the recovery phase, such as not scaling down immediately before tasks are about to expire; after scaling down, the intelligent network agent 130 continues to maintain the remote connection 172, preserving the premise for possible message-triggered recovery. In other words, the scaling down closed loop itself already contains preparation for the recovery closed loop, rather than simply being removed from the subsequent operational chain to release resources. It is this seamless connection that enables the AI ​​Agent workload 110 to remain recoverable even after being adjusted to zero replicas.

[0063] Depend on Figure 3 The process shown yields the following results: Before AI Agent workload 110 enters the zero-replica state, it undergoes combined judgment, continuous detection, and necessary time window constraints, thus preventing the scaling-down action from being directly triggered by a single momentary idle period; after AI Agent instance 111 exits operation, local connection 171 is interrupted while remote connection 172 remains, preserving the message access path in the system; the scaling-down action primarily affects the compute instance itself, without severing the external detection and remote connection conditions upon which recovery depends. This approach allows the resource release process and subsequent recovery process to be seamlessly integrated, rather than being isolated from each other.

[0064] Figure 4 This illustrates a process relationship for restoring a closed loop. After the AI ​​Agent workload 110 is in a zero-replica state, the system does not rely on manual intervention to restart the AI ​​Agent instance 111. Instead, the external scheduling controller 120 and the intelligent network agent 130 detect triggering events from different sources and call the workload control interface 160 when the recovery conditions are met, so that the number of replicas of the AI ​​Agent workload 110 is adjusted from 0 to a non-zero value. Figure 4The scheduled task trigger 401, external request trigger 402, message arrival trigger 403, replica count adjustment to non-zero 404, instance recovery 405, status loading 406, serviceability status determination 407, and business processing 408 together constitute a recovery path. This path, along with... Figure 3 The reduced-capacity closed loop is connected, with the former determining when to exit operation and the latter determining when to re-enter operation.

[0065] In this embodiment, recovery triggers can originate from different sources. Scheduled task trigger 401 is primarily obtained by the external scheduling controller 120 based on the scheduled task configuration 143. External request trigger 402 is primarily obtained by the external scheduling controller 120 after receiving and verifying an external request. Message arrival trigger 403 is primarily obtained by the intelligent network agent 130 after receiving a remote message on the remote connection 172. Among the three types of trigger events, scheduled task trigger 401 and external request trigger 402 are closer to planned or external call-type recovery, while message arrival trigger 403 is closer to instant message-type recovery. Although the detection entities for the three types of trigger events are different, they all ultimately act on the AI ​​Agent workload 110 through the workload control interface 160, thus exhibiting a consistent recovery action on the container orchestration platform 100 side.

[0066] Figure 9 This illustrates a timing relationship between scheduled task triggering and external request triggering. For scheduled task triggering 401, the external scheduling controller 120 can periodically read the scheduled task configuration 143, parse the scheduled triggering rules, and determine whether a task has reached its trigger time based on the current time. After the trigger time is reached, the external scheduling controller 120 does not directly replace the AI ​​Agent instance 111 in executing the task body. Instead, it first adjusts the number of replicas of the AI ​​Agent workload 110 through the workload control interface 160, causing the container orchestration platform 100 to recreate the AI ​​Agent instance 111. The reason for this is that the business entity executing the task is still the AI ​​Agent instance 111, and the external scheduling controller 120 mainly undertakes the responsibilities of detection and restart, rather than the business execution responsibilities.

[0067] For an external request triggering a 402 error, the external scheduling controller 120 can listen to a type of external call entry point. This external call entry point can be a Hypertext Transfer Protocol (HTTP) interface, a Remote Procedure Call (RPC) interface, or other entry points capable of handling external requests. Upon receiving an external request, the external scheduling controller 120 can first perform validity checks, such as verifying the access token, signature, request source, or path parameters. If the verification passes, the external scheduling controller 120 identifies the external request as an external request trigger event and calls the workload control interface 160 to restore the AI ​​Agent workload 110. If the external request itself carries business parameters, these parameters can be temporarily stored by the external scheduling controller 120 and then handed over for processing after the AI ​​Agent instance 111 reaches a serviceable state. This approach avoids directly exposing the business endpoint before the AI ​​Agent instance 111 has recovered, which could lead to request failure.

[0068] Figure 8 This illustrates a timing relationship where message arrival triggers recovery. The remote message service 150 sends a message to the intelligent network agent 130 via remote connection 172. Upon receiving the message, the intelligent network agent 130 first caches it using the message caching module 133, and then the recovery triggering module 134 calls the workload control interface 160 to adjust the replica count of the AI ​​Agent workload 110 from 0 to a non-zero value. The container orchestration platform 100 then creates a new AI Agent instance 111. Unlike scheduled task triggering 401 and external request triggering 402, message arrival triggering 403 is tightly coupled with the caching process because message arrival itself is instantaneous; without prior caching, the message may lose its receiving position before the AI ​​Agent instance 111 recovers. Figure 8 The timing sequence thus reflects the relationship that the message is first received, then recovery is triggered, and then it is forwarded after the instance reaches a serviceable state.

[0069] Regardless of the source of the recovery trigger, the workload control interface 160 plays a essentially consistent role during the recovery phase: adjusting the target replica count of the AI ​​Agent workload 110 from 0 to a non-zero value. In most scenarios, this non-zero value can be 1, as a single AI Agent typically requires only a single running instance to maintain business continuity. If the AI ​​Agent's business model allows for concurrent multi-instance processing, the non-zero value can be set to a value greater than 1. However, for one embodiment of this application, a more common scenario is single-instance recovery, where the replica count is adjusted from 0 to 1 to restore a single AI Agent instance 111. This approach makes it easier to guarantee the processing order of session state, task queues, and cached messages.

[0070] After the replica count is adjusted to non-zero (404), the container orchestration platform 100 enters the instance recovery (405) stage. Instance recovery (405) is typically not instantaneous but involves scheduling, container startup, volume mounting, network readiness, and application initialization. If the AIAgent workload 110 is implemented using Deployment, the container orchestration platform 100 can create a new Pod based on the image template; if it is implemented using StatefulSet, the recovery process may also include remounting stable volumes and rebuilding stateful identifiers. For the external scheduling controller 120 and the intelligent network agent 130, they typically do not need to concern themselves with all the details of instance recovery (405), but are primarily concerned with when the AI ​​Agent instance 111 can enter the subsequent state loading (406) and serviceability determination (407).

[0071] Figure 11 This illustrates the process relationship between state recovery and serviceability determination. After instance recovery 405 completes container-level startup, AI Agent instance 111 can read the main configuration file 141, authentication credentials 142, scheduled task configuration 143, session data 144, task queue 145, and plugin status 146 from persistent storage 140. This process corresponds to... Figure 11 The process involves configuration loading (501), authentication recovery (502), task configuration recovery (503), and session and task recovery (504). Layered recovery is used instead of a one-time, coarse read for two reasons. First, basic configuration and authentication information are prerequisites for subsequent access to other business states. Second, message processing and task execution have a stronger dependence on session data (144) and task queues (145). If business operations begin before these states are restored, context inconsistencies are likely to occur.

[0072] In one implementation, the main configuration file 141 and authentication credentials 142 can be restored first. The AI ​​Agent instance 111 can first determine its own identity information, access channel, and necessary authentication context before continuing to restore session and task-related data. After this process, even if the AI ​​Agent instance 111 needs to re-establish a local connection 171 with the intelligent network agent 130 in a later stage, it already possesses the necessary identity and authentication foundation. If the restoration of authentication credentials 142 relies on the platform key service, the AI ​​Agent instance 111 can also complete token refresh or credential verification at this stage, thereby reducing the probability of failure when processing cached messages later.

[0073] Task configuration recovery 503 is particularly important for the scheduled task trigger 401 scenario. Although the scheduled task trigger 401 is detected by the external scheduling controller 120, the actual task execution still requires the AI ​​Agent instance 111 to understand the task definition and execute the corresponding business. If the scheduled task configuration 143 is not recovered, even if the AI ​​Agent instance 111 recovers to the running state, it may only receive an abstract trigger signal and cannot obtain sufficient execution context. Session and task recovery 504 has a greater impact on message arrival trigger 403 and external request trigger 402 scenarios. Before cached messages are forwarded to the AI ​​Agent instance 111, the AI ​​Agent instance 111 usually needs to have sufficient session data 144 and task queue 145 in order to put new messages into the correct session context or map external requests to the correct business process.

[0074] The extent to which the 406 status loading is complete determines when AI Agent instance 111 can be considered serviceable. Figure 11 The local connection establishment error 505 and the serviceability status determination error 506 in the code reflect this issue. After the AI ​​Agent instance 111 resumes operation, it can re-establish the local connection 171 with the intelligent network agent 130. Once the local connection 171 is established, the intelligent network agent 130 logically has a channel to forward cached messages to the AI ​​Agent instance 111, but it is still not suitable to immediately forward all cached messages. The reason is that the establishment of the local connection 171 only proves that the link is reachable, and does not necessarily mean that the business status is sufficient to support message processing. Therefore, the serviceability status determination error 506 also needs to take into account the application-level preparedness.

[0075] In one embodiment, a serviceability status can satisfy at least several of the following conditions: the AI ​​Agent instance 111 process has started, the main configuration file 141 and authentication credentials 142 have been restored, the scheduled task configuration 143 has been loaded, the session data 144 and task queue 145 have at least completed basic recovery, the local connection 171 has been established, and the application-level health probe returns availability. The purpose of this combined determination is to distinguish between network reachability and service processability. If serviceability is determined solely based on port openness or container runtime status, cached messages may be prematurely forwarded to the AI ​​Agent instance 111, which does not yet have a session context, leading to message rejection, incorrect processing, or repeated entry into the recovery process.

[0076] In some implementations, the container orchestration platform 100 can provide a readiness probe as part of the serviceability status. The AI ​​Agent instance 111 can keep the probe returning "not ready" until basic state recovery is complete; the probe state is then switched after configuration loading (501), authentication recovery (502), and necessary session and task recovery (504). In addition to observing whether the service endpoint appears, the intelligent network agent 130 can also send probe messages to the AI ​​Agent instance 111 to determine whether the AI ​​Agent instance 111 truly possesses message processing capabilities. The probe message can be a lightweight handshake request or a pre-defined health check command. As long as the AI ​​Agent instance 111 returns a corresponding response, the intelligent network agent 130 can distinguish between the cached message forwarding phase and the simple network establishment phase.

[0077] The business processing 408 in the recovery loop is not limited to a single source. If the triggering event comes from message arrival trigger 403, then business processing 408 typically involves the intelligent network agent 130 forwarding cached messages or subsequent real-time messages to the AI ​​Agent instance 111, which then processes the messages in the recovered session context. If the triggering event comes from scheduled task trigger 401, then business processing 408 typically involves the AI ​​Agent instance 111 executing the corresponding task logic after the scheduled task configuration 143 recovers. If the triggering event comes from external request trigger 402, then business processing 408 typically involves the AI ​​Agent instance 111 processing request parameters temporarily stored or transferred by the external scheduling controller 120. While these three processes differ in their processing sources, they share the same constraint in the recovery chain: all must wait for the AI ​​Agent instance 111 to reach a serviceable state.

[0078] There is an engineering trade-off here. If the triggered service is sent directly to AI Agent instance 111 before the 406 state loading is complete, the apparent waiting time can be shortened, but the business logic and state recovery process will become entangled, making debugging more difficult and prone to intermittent failures. If the service is forwarded only after the 407 serviceability determination is completed, the recovery chain will be clearer, and error boundaries will be easier to converge. Therefore, in this embodiment, the recovery loop emphasizes restoring the instance first, then loading the state, then determining serviceability, and finally processing the service, rather than compressing these steps into a mixed process.

[0079] To illustrate how recovery latency affects the recovery strategy, the total instance recovery time can be defined in one embodiment. : (4) In equation (4) This indicates the time required for container orchestration platform 100 to create and start AI Agent instance 111. This indicates the time required for AI Agent instance 111 to load business state from persistent storage 140. This indicates the time required to establish local connection 171. This indicates the time required from the establishment of the local connection (171) to the determination of its serviceability status. For message arrival trigger 403, the retention time for the intelligent network agent (130) to cache messages should typically cover at least [the specified duration]. For the timed task trigger 401, the external scheduling controller 120 can also refer to the following when setting the window that is about to expire. This time split is not intended to mathematically represent the recovery process, but rather to illustrate the interrelationship between the scaling threshold, trigger window, and caching strategy.

[0080] In a specific scenario, if the remote messaging service 150 sends a message when the AI ​​Agent workload 110 has been scaled down to zero replicas, the intelligent network agent 130 can immediately receive and cache the message, simultaneously triggering the replica count to be adjusted from 0 to 1. After the container orchestration platform 100 creates the AI ​​Agent instance 111, the AI ​​Agent instance 111 first reads the main configuration file 141 and authentication credentials 142, then restores the session data 144 and task queue 145, and establishes a local connection 171 with the intelligent network agent 130. After the probe and probe messages jointly indicate that the AI ​​Agent instance 111 has reached a serviceable state, the intelligent network agent 130 forwards the message to the AI ​​Agent instance 111. If a second message arrives immediately afterward, the intelligent network agent 130 can continue to write the message to the cache queue, and forward it in a predetermined order after the AI ​​Agent instance 111 enters a stable processing phase. After this processing, although the AI ​​Agent workload 110 is in a zero-replica state before the message arrives, the message processing path remains continuous.

[0081] Figure 4 The result of the closed-loop recovery shown is that the AI ​​Agent workload 110 returns to the running state from a zero-replica state without manual intervention or the remote messaging service 150 re-identifying a new receiving subject. The external scheduling controller 120 handles the detection of scheduled task triggers 401 and external request triggers 402, the intelligent network agent 130 handles the detection of message arrival triggers 403, persistent storage 140 provides the necessary business state for recovery, and the container orchestration platform 100 executes instance recovery 405. With the participating entities sequentially linked, the AI ​​Agent instance 111 can resume operation and continue executing business functions at the appropriate time, without needing to run continuously during the entire idle period.

[0082] Figure 6 , Figure 7 and Figure 8 This collectively reflects the role of the intelligent network agent 130 in the message chain. In this embodiment, the intelligent network agent 130 is not merely a forwarding node between the AI ​​Agent instance 111 and the remote message service 150, but rather undertakes several actions including maintaining remote connections, receiving messages, caching messages, triggering recovery, and forwarding messages. Without this layer, if the AI ​​Agent instance 111 is scaled down, the original message access chain will be interrupted when the instance exits; and there will be no entity to receive messages before triggering recovery. Figure 7 The remote connection 172 and the local connection 171 thus correspond to two different types of connection responsibilities. The remote connection 172 is used to maintain the online relationship with the remote message service 150, while the local connection 171 is used to deliver cached or real-time messages to the AI ​​Agent instance 111 after it recovers.

[0083] When AI Agent instance 111 is running normally, messages sent by remote message service 150 can enter intelligent network agent 130 via remote connection 172, and then be transmitted to AI Agent instance 111 via local connection 171. At this time, intelligent network agent 130 can operate in either forwarding-first mode or light caching mode. Forwarding-first mode means that when AI Agent instance 111 is in a serviceable state and local connection 171 is normal, message receiving module 132 can directly hand over the received message to message forwarding module 135 for forwarding. Light caching mode means that even if AI Agent instance 111 is in a serviceable state, messages can still be briefly written to a cache structure before forwarding to cope with short-term fluctuations in local connection 171. The two methods do not conflict. In many engineering environments, storing messages in a memory cache before forwarding makes it easier to record message status and facilitates compensation processing when AI Agent instance 111 experiences short-term anomalies.

[0084] Figure 8The sequence of events shown more clearly illustrates the message processing path after the AI ​​Agent workload 110 has been scaled down to zero replicas. Messages from the remote message service 150 can no longer directly enter the AI ​​Agent instance 111 because the AI ​​Agent instance 111 does not exist at this time. The message receiving module 132 receives the message first, and the message caching module 133 then writes the message to the cache queue. After the write is completed, the recovery triggering module 134 calls the workload control interface 160 to drive the AI ​​Agent workload 110 to resume operation. Thus, there is no gap between message arrival and instance recovery; the message has already been received after entering the intelligent network agent 130. This link design makes the message arrival event itself both a business input and a recovery trigger signal.

[0085] The existence of a cache queue enables the message chain to have buffering capabilities. The cache queue is not limited to a specific implementation. In one embodiment, the cache queue can be implemented as a memory queue to store messages during a short recovery of the AI ​​Agent instance 111; in another embodiment, the cache queue can also be implemented as a local persistent structure with a write-before-write log to cover scenarios such as restarting the intelligent network agent 130 process or the AI ​​Agent instance 111 being offline for a long time. Figure 6 The message caching module 133 in the context of the application does not necessarily need to be understood as a single memory area; it can also include a memory caching layer and a persistent caching layer. The former handles low-latency paths, while the latter handles reliable paths. If the deployment scenario has a low tolerance for message loss, the message caching module 133 will typically prioritize a structure that can be written to disk or persisted.

[0086] In one implementation, message caching module 133 can generate a cache record for each message. The cache record can include at least a message identifier. Channel signage Reception time Message Status and message content summary Message identifier The preferred method is to use the unique message identifier provided by the remote messaging service 150; when the remote platform does not provide an explicit identifier, a message fingerprint can also be constructed based on the channel identifier, sender identifier, reception time, and content digest. Message Status It can be categorized into multiple states, such as received, cached, forwarded, and processed / confirmed. This processing ensures that the cache queue not only stores the message content but also the control information needed for subsequent recovery, forwarding, and deletion.

[0087] Message deduplication can be accomplished using the aforementioned cache records. If the remote message service 150 resends the same message due to network issues, or if a retransmission occurs during the recovery window, and the intelligent network agent 130 does not perform deduplication, the AI ​​Agent instance 111 may perform duplicate processing on the same business message. In one embodiment, the message caching module 133 can check the current message identifier before writing to the cache. The system checks if the message already exists in the deduplication index. If it does, the message is identified as a duplicate and refused to be written to the cache queue again. If it does not exist, the message is written to the cache queue and registered in the index. This index can be implemented using a hash table, key-value store, or sliding window Bloom filter. For AI Agent services, the need for strong consistency deduplication is related to deployment costs. If the platform prioritizes simplicity and stability, a hash index is more direct; if the platform prioritizes space efficiency, an approximate deduplication structure can be used, but this comes at the cost of a very low probability of false positives.

[0088] Sequence control is another type of constraint in message chains. Many AI agents deal with conversational message streams, where adjacent messages have semantic dependencies. If cached messages are forwarded out of order after being restored, although AI Agent instance 111 resumes operation, the processing result may have contextual bias. Therefore, the message caching module 133 can assign order information to each message when writing to the cache queue. The order information can be generated by a globally incrementing count or by generating it separately for each channel. In instant messaging scenarios, it is more common to maintain order within a channel while not requiring global ordering between channels. That is, messages in the same group, the same session, or the same business channel are ordered according to their arrival order, while parallel processing is allowed between different channels. This approach is more in line with the common semantics of conversational business and is also beneficial for reducing overall waiting time in multi-channel scenarios.

[0089] In one embodiment, messages belonging to the same channel in the cache queue can be sorted by sequence number. Forward in ascending order. If the first... The channel to which this message belongs is The highest sequence number currently confirmed for processing in this channel is Then the message forwarding module 135 can only forward messages that satisfy the following conditions: The next message. Waiting for AI Agent instance 111 to complete the corresponding processing and return confirmation. Moving forward further, this relationship can be represented as: (5) Equation (5) represents the first time... After the second confirmation, the channel The confirmed sequence number is advanced one unit. This advancement is predicated on the current message's sequence number being exactly equal to the previous confirmed sequence number plus one. Using this method, the intelligent network agent 130 can maintain a strict order for messages on the same channel during recovery. If the deployment scenario does not require such strict order constraints, batch confirmation or window confirmation methods can be used instead, but from a business continuity perspective, advancing message by message is easier to control.

[0090] Message Status Besides deduplication and order control, it also determines when cached records are deleted. If a message is deleted immediately after being written to the cache queue, it may be lost if the AI ​​Agent instance 111 exits abnormally during processing. If a message is not deleted for a long time after being forwarded, the cache queue will continue to grow. Therefore, the message caching module 133 and the message forwarding module 135 typically work together with a processing completion confirmation mechanism. This confirmation does not necessarily require a complex protocol; it only needs to be that the AI ​​Agent instance 111 can return the corresponding processing confirmation after completing message processing. Upon receiving the processing confirmation, the intelligent network agent 130 can change the corresponding message status from forwarded to processed and then remove the cached record through the cache cleanup process. If no processing confirmation is received within a preset time, the message status remains in the pending retry state or the forwarded pending confirmation state.

[0091] In engineering implementation, the four actions of message arrival, message caching, recovery triggering, and message forwarding should not be combined into a single atomic process. The message caching module 133 should preferably complete the write operation first, followed by the recovery triggering module 134 triggering the replica count adjustment. This is because if recovery is initiated before cache writing, messages may not have stably settled into the cache structure within the recovery time window of the AI ​​Agent instance 111. If the intelligent network agent 130 itself experiences an anomaly within this window, message reception will become incomplete. By prioritizing the cache write operation, messages first obtain a reliable destination, and then instance recovery is initiated. This processing chain facilitates troubleshooting and better meets the actual needs of message-driven recovery scenarios.

[0092] In one embodiment, the message caching module 133 can generate a cache completion flag after writing to the cache queue. The recovery trigger module 134 is only available when... Recovery actions are initiated only when the message is in a recoverable state. This is not to add extra complexity, but to ensure that replica switching is only allowed after the message has clearly entered a recoverable state. If the system relies solely on the order of function calls without explicitly distinguishing between the cache completion and recovery trigger phases, it is often difficult to pinpoint the fault location when troubleshooting message loss or duplicate recovery. By using a cache completion marker, the structural boundaries in the message chain become clearer.

[0093] Figure 8 The timing of post-recovery forwarding depends on whether AI Agent instance 111 has reached a serviceable state. Even if the intelligent network agent 130 has observed the appearance of the service endpoint corresponding to AI Agent instance 111, it should not immediately dump all messages in the cache queue onto AI Agent instance 111 at once. A more prudent approach is to determine whether it has the ability to process cached messages after the local connection 171 is established, based on application-level probe results, health probe results, or the ready flag actively returned by AI Agent instance 111. Figure 11 Serviceability status determination 506 and Figure 8 The message forwarding phase in this process is therefore directly related. The intelligent network agent 130 can maintain its cache queue unchanged until the serviceability status is determined, and then forward messages one by one or in batches after the determination is passed.

[0094] There is a connection between cached messages and real-time messages after recovery. If the remote message service 150 sends multiple messages continuously before the AI ​​Agent instance 111 has fully recovered, the message receiving module 132 can continuously write these messages into the cache queue. After the AI ​​Agent instance 111 reaches a serviceable state, the message forwarding module 135 can first forward the existing messages in the cache queue according to the order rules, and then switch to a hybrid forwarding mode of cached and real-time messages. The hybrid forwarding mode means that newly arrived messages can still be written into the cache queue first, but if the cached messages of the current channel have been digested to the end of the queue, the newly arrived messages can also be forwarded in near real-time. This processing can avoid a significant two-stage gap after recovery, that is, exhausting the historical cache before starting to receive new messages, which would cause real-time interaction to be stuck.

[0095] In some deployment scenarios, the message density of the remote message service 150 is high, and multiple cached messages may have accumulated within the recovery window. If these messages are forwarded sequentially globally, the initial processing latency of the AI ​​Agent instance 111 after recovery may be prolonged. If business semantics allow, the message forwarding module 135 can divide the message into multiple independent queues by channel, maintaining order within the same channel and forwarding in parallel between different channels. This parallelism does not negate order control, but rather limits order constraints to those with genuine semantic dependencies. For multi-group, multi-session, or multi-tenant scenarios, this implementation is more conducive to balancing order and throughput.

[0096] When deploying the intelligent network agent 130, the impact of its own anomalies must also be considered. If the intelligent network agent 130 restarts, and the cached data is only stored in the process memory, messages received before the AI ​​Agent instance 111 recovered may be lost. Therefore, when higher reliability is required, the message caching module 133 can write critical control information and message bodies to the persistence layer simultaneously. This persistence layer can be a local log file, an embedded database, or shared storage. After restarting, the intelligent network agent 130 can determine which messages have not yet been processed and acknowledged based on the cache record status and continue to retain these messages for subsequent forwarding. It is not required that the intelligent network agent 130 implement full message middleware-level capabilities, but at least a recoverable path for cached messages must be provided in typical failure scenarios.

[0097] from Figure 6 , Figure 7 and Figure 8 From the perspective of the reflected chain, the role of the intelligent network agent 130 is not abstract connection transport, but a set of structural actions to maintain message continuity during instance lifecycle switching. Remote connection 172 ensures that the remote message service 150 still has a stable access target; the cache queue ensures that messages do not lose their destination due to the absence of the AI ​​Agent instance 111; the recovery trigger module 134 ensures that message arrival can be transformed into instance recovery actions; and the message forwarding module 135 ensures that cached messages return to the business chain when the AI ​​Agent instance 111 is ready to process them. Breaking down these actions makes it easier to understand why the intelligent network agent 130 can support the scaling down of the AI ​​Agent workload 110 to a zero-replica operating mode.

[0098] The implementation described in this section yields the following results: messages from the remote message service 150 can still be received and cached when the AI ​​Agent instance 111 is not running; cached messages can be forwarded in the agreed-upon order after the instance recovers; duplicate messages can be identified and filtered before entering business processing; and the deletion of cached records does not depend on the instantaneous state when the message arrives, but is related to subsequent processing confirmation. This resulting link structure decouples the message channel from the runtime of the AI ​​Agent instance 111, allowing the AI ​​Agent workload 110 to more aggressively scale down during idle periods without shutting down message access capabilities.

[0099] Figure 10 and Figure 11The content shown more directly corresponds to the internal preparation process after the AI ​​Agent instance 111 is restored. After the AIAgent workload 110 is restored from a zero replica to a non-zero replica, the container orchestration platform 100 can complete Pod creation, image startup, and network access. However, these actions do not equate to the AI ​​Agent instance 111 having business processing capabilities. For this embodiment, the key to post-recovery operation is that the AI ​​Agent instance 111 needs to re-establish its running context from persistent storage 140, and only after this context reaches a processable level can it receive cached messages or process triggered business. If instance startup, state restoration, and business processing are mixed together, the error boundaries in the recovery chain will be unclear, which is not conducive to stable operation.

[0100] Figure 10 The main configuration file 141, authentication credentials 142, scheduled task configuration 143, session data 144, task queue 145, and plugin status 146 provided can be restored in layers according to their dependencies. The main configuration file 141 is usually located earlier because the AI ​​Agent instance 111 needs to clarify its own identity, access channel, basic parameters, and plugin activation information first. Authentication credentials 142 is also suitable for early restoration because some interactions after the local connection 171 is rebuilt, subsequent third-party calls, and some remote protocol negotiations may all rely on authentication information. Scheduled task configuration 143 can affect what specific business logic is executed after the scheduled task is triggered. Session data 144 and task queue 145 directly affect how cached messages and external requests are placed after restoration. Plugin status 146 is not the first state to block message processing in many scenarios, but if the AI ​​Agent instance 111 heavily relies on plugin collaboration, it can be moved to an earlier restoration stage.

[0101] In one implementation, after AI Agent instance 111 resumes operation, it can first perform basic loading, then business loading. Basic loading may include reading the main configuration file 141 and authentication credentials 142. Business loading may include reading scheduled task configuration 143, session data 144, task queue 145, and plugin status 146. One consideration for this separation is that the basic loading phase provides the prerequisites for AI Agent instance 111 to continue initialization, while the business loading phase provides the content that enables AI Agent instance 111 to restore business continuity. If session data 144 is read before basic loading is complete, common problems include incomplete configuration path preparation, uninitialized identity context, or unavailable authentication credentials 142, leading to failures in reading some business states. Separating basic loading from business loading results in a more stable recovery path.

[0102] Figure 11In the basic loading process, configuration loading (501) and authentication recovery (502) constitute the basic loading process. During this stage, AI Agent instance 111 can read the runtime parameters, channel definitions, role settings, and plugin enabling information from the main configuration file 141, and recover the credentials required to access the remote messaging service 150, external application interfaces, and key services from the authentication credentials 142. In some environments, authentication credentials 142 are not stored directly in plaintext, but are provided through the platform key service, encrypted mounting, or security tokens. After resuming operation, AI Agent instance 111 can first convert these credentials into an authentication context usable by the current session before continuing with subsequent loading. With this processing, if access to the database, object storage, or other state services is required later, it is not necessary to repeatedly wait for authentication preparation during the recovery process.

[0103] Task configuration recovery 503 can handle scenarios involving the recovery of scheduled task trigger events. While the external scheduling controller 120 is responsible for detecting whether a scheduled task has been triggered, the AI ​​Agent instance 111 still needs to know the specific task definition, execution entry point, and dependent context. If the scheduled task configuration 143 is not fully recovered, even if the AI ​​Agent instance 111 is already running, it may only be able to detect that a task has been invoked, but cannot determine the specific execution content of the task. Therefore, in scenarios where scheduled tasks account for a high proportion, the scheduled task configuration 143 should generally not be delayed too much. Some systems will place the scheduled task configuration 143 in the main configuration file 141 for unified management, while other systems will store it separately. Both methods are applicable, as long as the AI ​​Agent instance 111 has completed the corresponding configuration recovery before starting to execute the scheduled task.

[0104] The 504 error during session and task recovery primarily affects scenarios triggered by message arrival and external requests. If AI Agent instance 111 has not yet recovered session data 144 before forwarding a cached message to it, it's difficult to place the message back into the correct historical context. Task queue 145 also plays a crucial role. If pending tasks, tasks in progress, or results awaiting confirmation that existed before AI Agent instance 111's recovery are not reloaded after recovery, it may cause business omissions or lead to duplicate tasks being created after an external request is triggered. Therefore, task queue 145 is not only a list of tasks to be executed but also serves to handle subsequent business processes after recovery.

[0105] In one embodiment, the AI ​​Agent instance 111 can restore the business state in a preset order. Let's denote the restoration order of the main configuration file 141, authentication credentials 142, scheduled task configuration 143, session data 144, and task queue 145 as follows: , , , and Then it can satisfy: (6) Equation (6) represents the order of restoration, not a comparison of numerical values. Here... This indicates the recovery order of the main configuration file 141. Indicates the recovery order of authentication credential 142. This indicates the recovery order of scheduled task configuration 143. Indicates the recovery order of session data 144. This indicates the recovery order of task queue 145. This order is used because a later state is usually built upon the premise that a previous state is already available. If plugin state 146 is equally critical for connection establishment or message routing in the business scenario, it can also be inserted into the queue. and Between, or inserted into and between.

[0106] Recovery order does not equate to forced serialization. Certain states can be read concurrently under sequential constraints. For example, after AIAgent instance 111 completes the recovery of main configuration file 141 and authentication credentials 142, session data 144 and plugin state 146 can be read in parallel, and finally summarized into the current instance's runtime context by a unified state integration step. The purpose of this approach is not to shorten the apparent time consumption of each small step, but to avoid unnecessary serialization of dependent data. However, in this embodiment, even if partial concurrent recovery is allowed, the determination of whether a serviceable state has been reached is usually placed after all critical states have achieved basic availability.

[0107] Figure 11 The local connection establishment 505 occurs in the latter half of the state recovery process. After AI Agent instance 111 resumes operation, it can initiate a connection request to intelligent network agent 130, or intelligent network agent 130 can actively attempt to establish a local connection 171 after detecting the presence of AI Agent instance 111's service endpoint. Once local connection 171 is successfully established, the message chain is structurally closed, but whether the business chain is closed depends on whether AI Agent instance 111 has completed critical state recovery. Therefore, the establishment of local connection 171 can be considered one of the prerequisites for entering the serviceable state determination 506, rather than directly representing that cached messages can be consumed.

[0108] The serviceability status determination 506 can be achieved using a combination of multiple conditions. Simply relying on the container process starting is insufficient to determine whether session data 144 and task queue 145 have been restored. Similarly, simply relying on port connectivity is insufficient to determine whether the scheduled task configuration 143 has been loaded. A more robust approach is to incorporate container-level readiness, application-level state restoration, and link-level connectivity into the determination criteria. In one embodiment, the AI ​​Agent instance 111 can maintain a set of restoration status markers, including configuration restoration markers. Authentication recovery mark Task configuration restore marker Session recovery flag Task queue recovery marker and local connection establishment flag Once all the above markers are in an available state, then set AI Agent instance 111 to a serviceable state.

[0109] If used To indicate a serviceable status flag, the following relationship can be used: (7) In equation (7) This represents a logical AND operation. It is only used when a recovery flag is configured. Authentication recovery mark Task configuration restore marker Session recovery flag Task queue recovery marker and local connection establishment flag When both are true, Only then is it true. This expression does not require all systems to implement it as an explicit Boolean variable; it can also be expressed indirectly through an internal state machine, health check interface, or phased callbacks. The reason for using this combined judgment is that cached messages and triggered business logic truly depend on a set of states entering a processable range as a whole, rather than a single condition being true by chance.

[0110] In one embodiment, AI Agent instance 111 can determine its serviceability status by cooperating with a readiness probe and an internal state machine. The readiness probe indicates network and application visibility to the container orchestration platform 100, while the internal state machine indicates to the intelligent network agent 130 whether cached messages can be accepted. This two-layer decision-making helps address a common problem where the container orchestration platform 100 believes the instance is ready to join a service endpoint, but the intelligent network agent 130 still wants to wait for more business states to recover. This approach allows for timing control of service discovery and cached message forwarding at different granularities.

[0111] Once the service availability status check (506) passes, cached messages and triggered services enter the processing phase. For message arrival trigger events, the intelligent network agent 130 can retrieve the corresponding message from the cache queue and forward it to the AI ​​Agent instance 111 via local connection 171. For scheduled task trigger events, the AI ​​Agent instance 111 can directly enter the corresponding task execution entry point according to the scheduled task configuration 143. For external request trigger events, the request parameters temporarily stored by the external scheduling controller 120 can be handed over to the AI ​​Agent instance 111 for further processing. The use of a unified entry processing condition here ensures that services from different sources enter execution after reaching the same recovery threshold, thereby reducing behavioral differences between different trigger paths.

[0112] After cached messages enter processing, task queue 145 can also serve as a bridge between message-driven tasks and historical tasks. If AI Agent instance 111 had incomplete tasks before recovery, and receives new cached messages after recovery, the scheduling logic can decide whether to process historical tasks or new messages first based on task priority, session order, or trigger source. In session-based scenarios, many systems will prioritize processing cached messages within the same session before recovering subsequent tasks related to that session. In tool invocation scenarios, it can also first check whether there are any incomplete tool result confirmation processes in task queue 145 before deciding whether new messages should directly enter the main session process. This part belongs to the business scheduling strategy layer and does not change the overall structure of this embodiment regarding recovery order and serviceability status.

[0113] The recovery process also needs to handle a boundary case: the AI ​​Agent instance 111 receives a new external request or a new message arrival event before the state recovery is complete. If the system simply throws these events directly at the AI ​​Agent instance 111, which is not yet ready, concurrency conflicts can easily occur during recovery. A more appropriate approach is for the intelligent network agent 130 to continue writing new messages to the cache queue, and the external scheduling controller 120 to continue temporarily storing new requests in the external request buffer until the serviceability status determination 506 passes, at which point they are gradually released. In other words, the system can continue to receive input during recovery, but it is not in a hurry to feed this input directly to the business execution entity. With this approach, a buffer layer is formed between the input during recovery and the execution after recovery, making the entire operation smoother.

[0114] from Figure 10 and Figure 11As shown in the diagram, business state recovery is not about copying back the memory state byte by byte from where it stopped, but rather about restoring the AI ​​Agent instance 111 to a context state capable of continuing to process business. The main configuration file 141, authentication credentials 142, scheduled task configuration 143, session data 144, and task queue 145 correspond to different business dependencies such as identity, access, scheduling, context, and task continuation, respectively. After the local connection 171 is established, and by combining the serviceability status determination 506 with the control of cached messages and the timing of triggering business access, the recovered AI Agent instance 111 can re-enter the processing chain in a controlled manner, rather than being immediately exposed to all input traffic after startup.

[0115] The implementation described in this section yields the following results: After AI Agent instance 111 resumes operation, its business state can be restored in a dependent order. The restoration order and the serviceability status determination jointly determine the timing of cached messages and triggered services. Although local connection 171 is a prerequisite for message link restoration, it will not be mistakenly taken as the sole indicator that the service is ready to be processed. This state restoration structure allows for a smoother transition between instance restoration and service continuation, and also ensures that message triggering, scheduled task triggering, and external request triggering have more consistent processing boundaries after restoration.

[0116] Figure 12 This demonstrates a deployment implementation that can be directly implemented. The container orchestration platform 100 can respectively host the external scheduling controller Pod 1210, the intelligent network agent Pod 1310, and the AI ​​Agent Pod 1110. The AI ​​Agent Pod 1110 corresponds to the actual running carrier of the AI ​​Agent instance 111. The persistent volume 1410 can be used to host file-type states such as the main configuration file 141, scheduled task configuration 143, and plugin status 146, and can also serve as one of the persistent carriers for session data 144 or task queue 145. The service 1610 can provide a service discovery entry point for the AI ​​Agent Pod 1110, and can also serve as a network entry point for the intelligent network agent 130 or the external scheduling controller 120 to access the workload control interface 160. In this deployment mode, the external scheduling controller 120 and the intelligent network agent 130 can run continuously, while the number of replicas of the AI ​​Agent workload 110 can switch between 0 and non-zero.

[0117] The workload control interface 160 can be implemented in several ways. A straightforward approach is for the external scheduling controller 120 and intelligent network agent 130 to directly call the workload management interface provided by the container orchestration platform 100 to modify the target replica count of the AI ​​Agent workload 110. Another approach is to encapsulate a control service on the platform side, which uniformly receives scaling down and recovery requests and then interacts with the container orchestration platform 100. The latter adds an indirect call layer but offers better isolation and facilitates unified auditing and rate limiting in a multi-tenant platform. If the deployment environment already has a mature operation and maintenance control plane, using the encapsulated control service is more convenient; if the deployment environment prioritizes simplicity, directly calling the container orchestration platform 100's interface can also meet the requirements.

[0118] In one embodiment, the external scheduling controller 120 and the intelligent network agent 130 can be deployed as independent workloads. This approach has the advantage of clearly defined responsibilities, allowing scheduling detection and message channel processing to scale independently. If the message volume is large but the number of scheduled tasks is small, the intelligent network agent 130 can be scaled up separately; if there are many scheduled tasks and webhooks triggered but the message volume is low, the resources of the external scheduling controller 120 can be prioritized. Another deployment method is to combine the external scheduling controller 120 and the intelligent network agent 130 in the same Pod to reduce inter-component network calls. This approach is more suitable for small to medium-sized systems or single-tenant scenarios, but may not be conducive to independent scaling in large-scale platforms. Both deployment methods are applicable, as long as the scaling, recovery, state restoration, and message continuation links can be maintained.

[0119] The AI ​​Agent workload 110 can employ either a single-instance recovery mode or a controlled multi-instance mode. The single-instance recovery mode is more suitable for general AI Agent scenarios centered around a single session context, a single task queue, or a single identity channel. The controlled multi-instance mode is more suitable for scenarios with no strong state coupling, allowing parallel processing of different sessions or different business shards. However, in environments requiring sequential message processing, maintaining a single identity context, or strictly maintaining task queue consistency, the single-instance recovery mode is more reliable. This embodiment prefers the single-instance mode because message caching, session recovery, and task queue recovery introduce additional coordination costs in multi-instance parallel scenarios, and these coordinations are not necessary conditions for solving the current technical problems.

[0120] The system's storage implementation can also be configured differently according to business characteristics. If the session data 144 and task queue 145 are small, and simple deployment is the priority, persistent volume 1410 plus an embedded database is sufficient. If the data scale is large, or stronger retrieval and concurrent access capabilities are required, session data 144 and task queue 145 can be stored independently in an external database, while persistent volume 1410 mainly stores the main configuration file 141, scheduled task configuration 143, and plugin status 146. Authentication credentials 142 are generally not suitable to be directly scattered in general volumes, but are more suitable to be provided through a key management system, platform encrypted service, or dedicated credential mounting mechanism. The reason for this approach is not to change the main process of this embodiment, but to reduce the exposure surface of authentication information without changing the main process.

[0121] During system operation, the data flow between the external scheduling controller 120, the intelligent network agent 130, and the AI ​​Agent instance 111 can be summarized into three main lines. One main line is the control flow, where the external scheduling controller 120 and the intelligent network agent 130 drive the AI ​​Agent workload 110 to scale down and recover through the workload control interface 160. Another main line is the state flow, where the AI ​​Agent instance 111 reads business state from persistent storage 140 and writes newly added state back to persistent storage 140. The third main line is the message flow, where the remote message service 150 delivers messages to the intelligent network agent 130, and the intelligent network agent 130 delivers cached or real-time messages to the AI ​​Agent instance 111 at appropriate times. These three main lines are intertwined but not interchangeable. The control flow is responsible for the instance lifecycle, the state flow for the business context, and the message flow for message access and delivery. Processing these three flows separately helps the system maintain a clear structure during scaling down and recovery.

[0122] Figures 1 to 12 The illustrated scheme can also cover several different engineering implementations. The remote message service 150 is not limited to a specific platform; as long as there is a persistent connection or a persistent event push relationship, it can serve as a remote message source. The external request trigger 402 is not limited to HTTP requests; message queue callbacks, event bus callbacks, or other platform notification methods can all be used as implementations of external requests. The workload control interface 160 is not limited to a specific platform interface protocol; as long as it can switch the number of replicas of the AI ​​Agent workload 110 between zero and non-zero replicas, it can be included in the implementation scope of this embodiment. The persistent storage 140 is not limited to a single database, a single file system, or a single volume mounting method; it can be composed of multiple storage mechanisms. This approach can cover common equivalent implementations on the platform side without changing the technical essence of this embodiment.

[0123] In one embodiment, the system can also set up runtime logs related to scaling down and recovery. The external scheduling controller 120 can record the time of each idle detection, the idle determination result, the scaling down execution time, and the source of the recovery trigger. The intelligent network agent 130 can record message reception time, cache write time, recovery trigger time, message forwarding time, and processing confirmation time. The AI ​​Agent instance 111 can record the state loading time, the establishment time of the local connection 171, and the time when the serviceable state is achieved. These logs are not necessarily essential components of system operation, but they are valuable in actual deployments. When slow recovery, duplicate recovery, or message delays occur, it is possible to more quickly pinpoint whether the problem originates from replica creation, state loading, or the message chain. Using these logs does not change the business logic but reduces debugging costs.

[0124] At the system entity level, it can be... Figure 1 The object shown is implemented as an on-demand scaling and automatic recovery system. This system includes an AI Agent workload 110, an external scheduling controller 120, an intelligent network agent 130, and persistent storage 140. The AI ​​Agent workload 110 runs the AI ​​Agent instance 111; the external scheduling controller 120 detects idle conditions, scheduled task trigger events, and external request trigger events, and controls the number of replicas of the AI ​​Agent workload 110 through the workload control interface 160; the intelligent network agent 130 maintains a remote connection 172 with the remote message service 150 after the AI ​​Agent instance 111 terminates, receives messages, caches messages, and initiates recovery when a message arrival trigger event is detected; persistent storage 140 stores the business state required for the AI ​​Agent instance 111 to resume operation. This system entity can be represented as a distributed, multi-component system or as a runtime platform implemented by multiple services.

[0125] Regarding the storage medium, this embodiment can also be implemented as a computer-readable storage medium. After the computer program is stored on this storage medium and executed by the processor, it can perform operations such as idle condition detection, workload replica count adjustment, remote connection maintenance, message caching, recovery triggering, service status loading, and service availability determination. The processor here can reside in an external scheduling controller 120, an intelligent network agent 130, a platform control node, or other computing devices that carry these functions. Since many actions in this embodiment can be implemented through program logic, it is feasible to use a storage medium to carry the relevant control logic. This implementation does not require all program logic to be executed centrally on a single processor; it can also be distributed across multiple processors or multiple service instances to complete collaboratively.

[0126] Figures 1 to 12The illustrated embodiment forms a relatively complete engineering chain. The AI ​​Agent workload 110 enters a zero-replica state under the determination of the external scheduling controller 120. The intelligent network agent 130 maintains the remote connection 172 and receives remote messages after the AI ​​Agent instance 111 terminates. The external scheduling controller 120 and the intelligent network agent 130 drive the AI ​​Agent workload 110 to resume operation under different event triggers. After recovery, the AI ​​Agent instance 111 reads the business status from persistent storage 140, then reconnects to the message link via local connection 171 and continues processing business. Each structure in this chain corresponds to a specific action, and each action is dependent on the preceding and following links. Because these dependencies are clearly assigned to different entities, the AI ​​Agent instance 111 does not need to remain continuously resident during the entire idle period.

[0127] The following technical results can be obtained from this embodiment: the AI ​​Agent workload 110 can be scaled down to zero replicas during idle periods, thereby reducing the consumption of computing and memory resources by the running instance; after the AI ​​Agent instance 111 exits operation, the remote connection 172 can still be maintained by the intelligent network agent 130, and the message access path will not directly fail due to the instance exit; the scheduled task trigger event, the external request trigger event, and the message arrival trigger event can respectively drive the AI ​​Agent workload 110 to resume operation, and the recovered AI Agent instance 111 can also continue to process cached messages or other services based on the business state in the persistent storage 140. This structural chain enables resource release, connection maintenance, trigger recovery, and service continuation to coexist collaboratively in the same system.

[0128] It should be understood that the method steps in the foregoing embodiments can be implemented by program instructions controlling related hardware, or by dedicated circuits, programmable logic devices, or a combination thereof. Correspondingly, the systems, devices, modules, units, or components in the foregoing embodiments can be implemented in software, hardware, or a combination of both. The division of modules, units, or components is merely a logical division for the purpose of illustrating the technical solution; in actual implementation, they can be combined, split, or integrated as needed.

[0129] In one embodiment, the electronic device may include a processor, a memory, and a communication interface, wherein the memory is used to store program instructions, and the processor is used to call and execute the program instructions to implement all or part of the steps in the foregoing method embodiments. The electronic device may be a server, a terminal device, an edge computing node, a cloud computing device, or other device with data processing capabilities.

[0130] In one embodiment, this application may also be implemented in the form of a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, causes the processor to implement all or part of the steps in the foregoing method embodiments. The computer-readable storage medium may be a read-only memory, random access memory, flash memory, hard disk, solid-state drive, optical disk, or other non-transitory storage medium.

[0131] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, system embodiments are basically similar to method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. In the description of this specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0132] Furthermore, the terms "including," "comprising," and "having" used in the specification are all non-exclusive inclusions; the terms "first," "second," etc., are only used to distinguish technical features and do not indicate limitations on order, quantity, or importance. The execution order of each step in the method embodiments is also not absolutely limited. Without departing from the technical concept of this application, the steps can be adjusted in order, executed in parallel, combined, or split for execution.

Claims

1. A method for on-demand scaling and automatic recovery of AI Agents, applied to a container orchestration platform, wherein the container orchestration platform deploys AI Agent workloads, an external scheduler controller, an intelligent network agent, and persistent storage, the method comprising: When the external scheduling controller detects that the AI ​​Agent workload meets the idle condition, it calls the workload control interface of the container orchestration platform to adjust the number of replicas of the AI ​​Agent workload to 0, thereby terminating the operation of the AI ​​Agent instance; After the AI ​​Agent instance terminates, the intelligent network agent maintains a remote connection with the remote messaging service and receives messages sent by the remote messaging service. The external scheduling controller detects timed task trigger events or external request trigger events, and / or the intelligent network agent detects message arrival trigger events. Upon detecting any trigger event, the workload control interface is invoked to adjust the number of replicas of the AI ​​Agent workload to a non-zero value in order to restore the operation of the AI ​​Agent instance. After the AI ​​Agent instance resumes operation, the AI ​​Agent instance loads the business state from the persistent storage and processes the cached message corresponding to the message arrival trigger event and / or the business corresponding to the scheduled task trigger event or the external request trigger event.

2. The method of claim 1, wherein, The idle conditions include the following: There are no pending tasks; There are no pending external trigger requests; There are no cached messages to be forwarded; There are no scheduled tasks that are about to expire.

3. The method of claim 1 or 2, wherein, When the external scheduling controller detects that the AI ​​Agent workload meets the idle condition, it calls the workload control interface of the container orchestration platform to adjust the replica count of the AI ​​Agent workload to 0, including: The idle condition is detected by the external scheduling controller according to a preset inspection cycle; When multiple consecutive test results meet the idle condition, the workload control interface is invoked to adjust the number of replicas of the AIAgent workload to 0.

4. The method of claim 1, wherein, After the AI ​​Agent instance terminates, the intelligent network agent maintains a remote connection with the remote messaging service and receives messages sent by the remote messaging service, including: Maintain the remote connection between the intelligent network agent and the remote messaging service; Disconnect the local connection between the AI ​​Agent instance and the intelligent network agent; When a message is received from the remote messaging service, the message is cached.

5. The method of claim 1 or 4, wherein, The intelligent network agent detects message arrival trigger events and, upon detecting such events, invokes the workload control interface to adjust the number of replicas of the AI ​​Agent workload to a non-zero value, thereby resuming the operation of the AI ​​Agent instance. This includes: After receiving a message from the remote messaging service, the message is written into the cache queue; The workload control interface is invoked based on the message arrival trigger event to resume the operation of the AI ​​Agent instance; Once the AI ​​Agent instance reaches a serviceable state, the messages in the cache queue are forwarded to the AI ​​Agent instance.

6. The method of claim 1, wherein, The external scheduling controller detects scheduled task trigger events or external request trigger events, including: Read the scheduled task configuration and parse the corresponding scheduled trigger rules to determine the scheduled task trigger event; Receive external requests and perform validity checks to determine the event triggered by the external request.

7. The method of claim 1, wherein, After the AI ​​Agent instance resumes operation, the AI ​​Agent instance loads the business state from the persistent storage, including: Load the main configuration file and authentication credentials; Load the scheduled task configuration; Load session data and task queue.

8. The method of claim 1 or 7, wherein, After the AI ​​Agent instance resumes operation, the AI ​​Agent instance loads the business state from the persistent storage and processes the cached message corresponding to the message arrival trigger event and / or the business corresponding to the scheduled task trigger event or the external request trigger event, including: The AI ​​Agent instance establishes a local connection with the intelligent network agent; After the AI ​​Agent instance reaches a serviceable state, it processes the cached messages and / or the business corresponding to the scheduled task triggering event or the external request triggering event.

9. An on-demand scaling and automatic recovery system for AI Agents, applied to a container orchestration platform, comprising: AI Agent workload, used to run AI Agent instances; An external scheduling controller is used to call the workload control interface of the container orchestration platform to adjust the number of replicas of the AI ​​Agent workload to 0 when the AI ​​Agent workload is detected to meet the idle condition, and to call the workload control interface to adjust the number of replicas of the AI ​​Agent workload to a non-zero value when a scheduled task trigger event or an external request trigger event is detected. The intelligent network agent is used to maintain a remote connection with the remote message service after the AI ​​Agent instance terminates, receive messages sent by the remote message service, and call the workload control interface when a message arrival trigger event is detected to adjust the number of replicas of the AI ​​Agent workload to a non-zero value. Persistent storage is used to save the business state required for the AI ​​Agent instance to resume operation; The AI ​​Agent instance loads the business state from the persistent storage after resuming operation, and processes cached messages and / or the business corresponding to the scheduled task trigger event or the external request trigger event.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of claims 1 to 8.

Images