Fault-tolerant scheduling method and system for model services

Abstract

The embodiment of the application provides a fault-tolerant scheduling method and system of a model service, failure feature data in scheduling failure information is extracted; a preset error classifier is used to determine an error category corresponding to the scheduling failure information, at least containing one of a transient error type and a persistent error type; according to the error category, an execution flow of an intelligent agent scheduling task is mapped to a same model retry path or a model switching path; if the execution flow is mapped to the model switching path, a failed model set recorded in metadata of the intelligent agent scheduling task is read, a first candidate model not belonging to the failed model set is searched from candidate models for scheduling switching, and a failure model of the scheduling is added to the failed model set. By accurately separating transient errors and persistent errors, differential responses are performed to balance retry costs and switching costs, and it is ensured that the scheduling routing does not fall into a dead loop.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and distributed task scheduling technology, specifically to a fault-tolerant scheduling method and system for model services. Background Technology

[0002] In agent-based automated scheduling systems using Large Language Models (LLM), the uncontrollability of remote model services (such as network jitter, service rate limiting, quota restrictions, and backend service overload) is a key factor affecting the overall stability of the system. In a production environment, how these scheduling anomalies are handled directly impacts the system's execution efficiency and operating costs. Currently, the mainstream solutions for handling scheduling anomalies in the industry mainly fall into two categories: One approach uniformly retryes the same model using an exponential backoff algorithm for all scheduling failures; the other tends to immediately change the scheduling target upon detecting anomalies. The former is prone to deadlock due to ineffective, continuous retries when irreparable logical errors occur. The latter, with its overly sensitive switching mechanism, is prone to misinterpreting brief link fluctuations or single-instance pressure as model service unavailability, leading to frequent model switching and causing drastic fluctuations between high-quality models and high-cost or low-efficiency backup models, impacting inference quality. A common drawback of both approaches is the lack of fine-grained error identification; they cannot distinguish between errors where "retrying might succeed" and "retrying will be futile," nor can they differentiate between the granularity of "the model is currently failing as a whole" and "this particular scheduling failure is an occasional anomaly."

[0003] Therefore, how to make differentiated responses to failures in an agent scheduling system that schedules unstable remote large language model services in order to achieve a balance between retry costs and switching costs has become a technical problem that urgently needs to be solved in the current field. Summary of the Invention

[0004] In view of this, embodiments of this application provide a fault-tolerant scheduling method, apparatus, and related equipment for model services to at least partially solve the above problems.

[0005] In a first aspect, embodiments of this application provide a fault-tolerant scheduling method for a model service, including: Obtain scheduling failure information and extract failure feature data from the scheduling failure information; the scheduling failure information is generated when the remote large language model service fails to schedule. Using a preset error classifier based on the failure feature data, the error category corresponding to the scheduling failure information is determined, and the error category includes at least one of transient error type and persistent error type; Based on the error category, the execution flow of the current agent scheduling task for the remote large language model is mapped to the same model retry path or the model switching path. If the execution process is mapped to the model switching path, the set of failed models recorded in the metadata of the agent scheduling task is read, the first candidate model that does not belong to the set of failed models is retrieved from the preset global candidate models and the model is switched, and the remote large language model before the switch is added to the set of failed models as a fault model.

[0006] Secondly, based on the fault-tolerant scheduling method for model services described in the first aspect of this application, embodiments of this application also provide a fault-tolerant scheduling system for model services, including: The acquisition module is used to acquire scheduling failure information and extract failure feature data from the scheduling failure information; the scheduling failure information is generated when the remote large language model service fails to schedule. The classification module is used to determine the error category corresponding to the scheduling failure information based on the failure feature data using a preset error classifier. The error category includes at least one of transient error type and persistent error type. The mapping module maps the execution flow of the current agent scheduling task for the remote large language model to the same model retry path or model switching path according to the error category. The switching module is used to read the set of failed models recorded in the metadata of the agent scheduling task if the execution process is mapped to the model switching path, retrieve the first candidate model that does not belong to the set of failed models from the preset global candidate models, switch the model, and add the remote large language model before the switch as the fault model to the set of failed models.

[0007] Thirdly, embodiments of this application also provide a computer storage medium storing computer-executable instructions, wherein when the computer-executable instructions are executed, a fault-tolerant scheduling method for any model service as described in the first aspect of embodiments of this application is executed.

[0008] This application provides a fault-tolerant scheduling method and system for model services. The method extracts failure feature data from acquired model scheduling failure information; uses a preset error classifier to determine the error category corresponding to the scheduling failure information based on the failure feature data, where the error category includes at least one of transient and persistent error types; according to the error category, maps the execution flow of the current agent scheduling task for the remote large language model to a retry path or a model switching path; if the execution flow is mapped to the model switching path, it reads the set of failed models recorded in the metadata of the agent scheduling task, retrieves the first candidate model not belonging to the set of failed models from a preset global candidate model list, performs a scheduling switch, and adds the remote large language model before the switch as a fault model to the set of failed models. This method uses an error classifier to match the failure feature data extracted from the scheduling failure information, accurately separating transient and persistent errors. For transient errors, priority is given to mapping to the same model's retry path for backoff, preserving the business advantages of the preferred large model in terms of response quality and performance to the greatest extent, and significantly reducing switching costs; for persistent errors, direct route switching is performed to avoid meaningless retries, greatly saving billing costs and waiting processing latency. It also effectively ensures that routing does not get stuck in an infinite loop. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0010] Figure 1 A schematic diagram illustrating the workflow of a fault-tolerant scheduling method for a model service provided in this application embodiment; Figure 2 A schematic diagram of a retry waiting time interval control process provided in an embodiment of this application; Figure 3 A flowchart illustrating a dynamic recovery control strategy provided in an embodiment of this application; Figure 4 A flowchart illustrating an execution cycle scheduling task provided in an embodiment of this application; Figure 5 A schematic diagram of the structure of a fault-tolerant scheduling system for model services provided in this application embodiment; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0011] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art should fall within the protection scope of the embodiments of this application.

[0012] It should be understood that the steps described in the method embodiments of this application may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this application is not limited in this respect.

[0013] Example 1 This application provides a fault-tolerant scheduling method for model services, such as... Figure 1 As shown, Figure 1 This paper illustrates a flowchart of a fault-tolerant scheduling method for a model service provided in an embodiment of this application, including: Step S101: Obtain scheduling failure information and extract failure feature data from the scheduling failure information; the scheduling failure information is generated when the remote large language model service fails to schedule. In this embodiment, the model service refers to a large language model (LLM) inference service deployed on a remote network and provided asynchronously or synchronously through a standard application programming interface (API). During operation, this service may experience unpredictable service scheduling response failures due to service demand overload, concurrent traffic congestion, or quota policies. The scheduling failure information is the response failure error data packet or physical connection timeout message generated at the interface layer when the large model service provider, as an external service, is requested for service scheduling by the user end due to reasons similar to those mentioned above. In the actual application scenario of this embodiment, the service interfaces of various large language model service providers (such as OpenAI, Deepseek, or enterprise internal gateways) are different, and the corresponding error message formats vary greatly. If such non-standard messages are directly parsed, complex routing code logic needs to be configured, and the stability of the interface changes of the interfacing vendors is extremely poor. Therefore, this implementation first establishes a unified fault information collection process, which obtains feature data of scheduling failure information through feature extraction as the basis for subsequent processing, so as to achieve rapid cleaning and standardization of heterogeneous error information, block the logical interference of heterogeneous cloud service errors on the scheduling engine, and thus improve the compatibility of the method described in this implementation.

[0014] Step S102: Using a preset error classifier based on the failure feature data, determine the error category corresponding to the scheduling failure information. The error category includes at least one of transient error type and persistent error type. In the actual application scenario of this embodiment, different scheduling failure reasons are vastly different in physical mechanism. For example, rate limiting is a transient fault at the second level, which can be recovered by simply waiting for a period of time and then retrying; while such as service interruption due to unpaid fees or API key expiration are hard persistent errors, and even if retrying multiple times, it is just a waste of resources. Traditional solutions do not distinguish between "transient" and "persistent" fault attributes, resulting in either the user execution end blindly retrying in place or directly switching to a usable high-quality target model, both of which lead to a poor user scheduling experience. Therefore, this step in this embodiment limits the input of the obtained structured scheduling failure feature data into a preset error classifier. This error classifier can perform feature scanning corresponding to different attribute features, thereby quickly classifying the causes represented by the scheduling failure information and determining whether the scheduling failure error category belongs to transient error type or persistent error type, so as to provide a reliable basis for subsequent scheduling decisions.

[0015] Step S103: Based on the error category, map the execution flow of the current agent scheduling task for the remote large language model to either the same-model retry path or the model switching path. This embodiment overcomes the drawback of the relatively fixed switching mode in traditional solutions by using this step, finding a better balance between the "delay waiting" of retrying and the "quality loss cost" of switching models. Specifically, if the scheduling failure is identified as a transient error caused by rate limiting or network timeout, the scheduling execution flow is controlled to flow to the same-model retry path. This allows for a delay and waiting for the cloud concurrency peak of the large model to subside before scheduling it again, without replacing the currently selected high-quality target model. Conversely, if the current scheduling failure is identified as a persistent error such as quota exhaustion, the scheduling execution flow is forced to flow to the model switching path, abandoning the unschedulable faulty large model and initiating cross-node scheduling of other available large models to execute the current scheduling task. This embodiment achieves a better benefit ratio by using a precise differentiated response mechanism, which enables transient faults to be executed smoothly on the same model, while persistent faults are bypassed in an instant. Under the dual constraints of low system latency and API billing costs, this improves the user's model scheduling experience.

[0016] Step S104: If the execution process is mapped to the model switching path, then read the set of failed models recorded in the metadata of the agent scheduling task, retrieve the first candidate model that does not belong to the set of failed models from the preset global candidate models, perform scheduling switching, and add the remote large language model before switching as the fault model to the set of failed models.

[0017] In traditional technical solutions, scheduling failure switching decisions occur at the scheduling level, generally without retaining task execution state memory. If a backup large model experiences continuous failures for a specific scheduled task due to context limitations or other functional constraints, the system, lacking task-level state memory, may repeatedly attempt to schedule a model already confirmed as unschedulable in each subsequent retries. This leads to repeated consumption of scheduling resources and can cause the task to enter a routing loop. To avoid this, this implementation limits this step to reading the currently available global candidate model list when the process flows to the model switching path, and extracting the set of failed models from the metadata of the current specific scheduled task using deserialization from the state memory database. The set of failed models is used as an exclusion set to filter the global candidate model list, removing large models that have failed during the current task execution period. The first high-priority large model not belonging to the exclusion set is selected as the target candidate model, the current routing pointer is redirected to this model for switching, and the remote large language model before the switch is appended to the set of failed models. This endows the fault-tolerant scheduling system of this embodiment with "failure avoidance memory" at the task lifecycle level. It ensures that the same specific task will never repeatedly collide with a failed node, perfectly breaking the pain point of routing dead loops and quota waste caused by repeated collisions that are easy to fall into in traditional chain-type failover.

[0018] Optionally, in one implementation of this embodiment, the method further includes: accumulating the number of retries for the agent scheduling task; when the agent scheduling task is a periodic task triggered at a time, and the accumulated number of retries reaches a preset maximum retry threshold, the current state of the periodic task is converted to a reset state, so that the agent can be rescheduled to execute the periodic task when the next triggering cycle arrives. In this embodiment, periodic tasks (such as timed financial reports, daily synchronized data) and instant scheduling tasks have fundamentally different lifecycle control after failure. Specifically, instant task scheduling failure can be directly marked as a failure and a work order can be manually dispatched. However, for periodic tasks, if the retry limit is reached during the peak of a sudden failure (such as a sudden large-scale network outage in the cloud) and it is simply marked as a "persistent failure", it will not only trigger a large-scale system shutdown alarm risk, but also affect the system's self-healing ability to recover when the scheduling environment returns to normal. Therefore, in this embodiment, this stage is limited to creating a counter register in the metadata to accumulate the number of scheduling retries during the current agent (model) scheduling task runtime. If the current task is determined to have a timed trigger attribute, and the accumulated count reaches the preset maximum retry threshold, the system's scheduling state machine can intervene and disable the failure flag, forcibly converting the current periodic task's state flow to a reset state. In this reset state, the number of retries corresponding to the task is cleared, and its associated set of failed models is also removed. The task is then placed in the idle waiting queue, silently waiting for the next Cron periodic trigger time. This configures a time window for the periodic scheduled task to resume normal operation. Therefore, in the event of a large-scale severe network failure or service paralysis, the system implementing this solution can wait for environmental self-healing in an unattended environment, ensuring smooth availability when the next scheduling trigger is deployed, and reducing the stability interference caused by short-term network failures to the system implementing this method.

[0019] Optionally, in one implementation of this embodiment, obtaining scheduling failure information and extracting failure feature data from the scheduling failure information includes: intercepting large language model interface requests at the network transport layer or large language model interface middleware layer, obtaining heterogeneous raw error data packets or underlying connection exception error information returned by the remote large language model service scheduling failure, as the scheduling failure information; parsing and extracting the corresponding Hypertext Transfer Protocol status code, error message text, and raw error type field defined by the large model vendor from the scheduling failure information as the failure feature data. In the complex distributed cloud-native architecture of this embodiment, the generation of exception packets may cross the application layer HTTP gateway, or even cause the physical TCP of the transport layer to be disconnected. Therefore, it is necessary to establish a physical catcher with high sensitivity and strong protocol generalization. Therefore, this step in this embodiment limits the deployment of a low-level feature catcher at the large language model user-end socket network connection pool layer or interface gateway interception middleware layer. When an API request causes a socket timeout or connection rejection due to network failure, or when a returned error data packet is obtained, the catcher intercepts it as raw scheduling failure information. The intercepted failure information data packet is then parsed, and the corresponding Hypertext Transfer Protocol (HTTP) status code, message text (such as the text string under error.message), or service vendor-defined underlying identifier type fields are extracted as failure feature data and passed forward. This process quickly maps and captures unstructured physical anomaly information such as network disconnections and gateway timeouts scattered throughout the underlying physical protocol stack into uniformly formatted, protocol-decoupled failure feature data, eliminating the impact of external interface and communication changes on the scheduling layer.

[0020] Optionally, in one implementation of this embodiment, determining the error category corresponding to the scheduling failure information based on the failure feature data using a preset error classifier includes: configuring the preset error classifier to perform semantic analysis on the failure feature data based on a machine learning text classification model, or using a large language model as a classifier, to perform multi-dimensional feature recognition on the scheduling failure information and determine the error category corresponding to the scheduling failure information. In the practical application scenario of this embodiment, the error text updates from model service providers are often quite frequent. Traditional hard-coded regular expression matching strategies are difficult to adapt to unknown, new large model interface information text. For example, when a vendor fine-tunes "quota exceeded" to "credit limit reached," the error category identification method using traditional regular expression matching will fail. Therefore, this embodiment introduces intelligent text classification technology with high semantic generalization and autonomous reasoning capabilities for error category identification in this step. Specifically, for example, a lightweight vectorized semantic classification model (such as a pre-trained SVM text classification model or a lightweight bidirectional encoder word vector classification network) can be configured in the error classifier, or the large model itself can be used as the arbiter (LLM-as-classifier). By sending a request with the classification rule prompt and the failure message text to be detected to the local lightweight large model, the large model extracts the deep semantics of the error text, identifies and finally classifies and outputs the corresponding error category. This breaks through the mismatch blind spot caused by traditional static rule matching, endows classification matching with intelligent reasoning capabilities in the semantic recognition dimension, and achieves extremely high robustness in recognizing new and variant error texts.

[0021] Optionally, in one implementation of this embodiment, determining the error category corresponding to the scheduling failure information based on the failure feature data using a preset error classifier includes: matching the error message text in the failure feature data with a preset set of regular features to obtain a matching result; determining the error category of the scheduling failure based on the matching result, wherein the category includes at least one of quota exhaustion category, rate limiting category, model overload category, network timeout category, identity authentication error category, and general error category; wherein the quota exhaustion category and the identity authentication error category belong to the persistent error type, and the rate limiting category, the model overload category, and the network timeout category belong to the transient error type.

[0022] In the practical application scenario of this embodiment, when processing known routine fault text, a classification logic based on regular expression matching with minimal computational overhead can be adopted. This classification logic can ensure that the classification response latency remains at the microsecond level, guaranteeing that a large number of intelligent agent workflows will not introduce any additional latency load when misclassified. Specifically, the error message text and HTTP status code can be extracted from the failure feature data by the error classifier. Then, the extracted information is matched against a preset regular expression table in sequence: for example, matching "quota exceeded" and HTTP 402 / 403 is classified as quota exhausted; matching "rate_limit_exceeded" or HTTP 429 is classified as rate limiting; matching "overloaded" or HTTP 503 is classified as model overload; matching network timeout or HTTP 504 is classified as network timeout; matching "invalid_api_key" or HTTP 401 is classified as authentication error. For those that are not matched, the configured general error is used as a fallback. This step utilizes a low-power, nanosecond-level parallel regularized scanning matching mechanism to quickly identify persistent errors such as quota exhaustion and transient errors such as current limiting, providing efficient and low-latency reliable input for subsequent mapping decisions.

[0023] Optionally, in one implementation of this embodiment, mapping the execution flow of the current agent scheduling task for the remote large language model to a retry path or a model switching path based on the error category includes: When the error category is the quota exhaustion category or the model overload category, the current execution flow will be mapped to the model switching path; When the current error category is the rate limiting category or the network timeout category, the current execution flow will be mapped to the same model retry path; When the current error category is the identity authentication error category, a preset alarm policy is triggered and the current agent scheduling task is suspended to wait for external operation and maintenance intervention. When the current error category is the general error category, the current execution flow is mapped to the same model retry path, and a conservative retry decision with a limited number of retries is made.

[0024] In this embodiment, the above steps define a differentiated routing strategy for scheduling tasks based on the physical attributes of different error categories, in order to protect the hit rate of the preferred large model scheduling and avoid the waste of worthless retry quotas. Specifically, this embodiment will exemplarily describe the above steps here. When mapping the execution flow of the current agent scheduling task for the remote large language model to the same model retry path or model switching path according to the error category, for example, it can be based on the received category label and retrieve a preset mapping routing table, which includes at least the following branches: Branch 1: When the failure is due to persistent paralysis such as quota exhaustion or model overload, the process will switch to the model switching path and abandon the model. Branch 2: When the congestion is due to transient events such as rate limiting or timeout, the process will be redirected to the same model retry path, waiting for the cloud concurrency peak to subside. Branch 3: When the error is a hard authentication error, the scheduling task is suspended and an alarm is triggered to notify the operations team to intervene and prevent pointless collisions. Branch 4: When it is a general error, it will flow to the same model retry path and limit the execution to a conservative retries of up to 2 times.

[0025] The implementation of the above steps effectively avoids meaningless scheduling retries and overly sensitive scheduling switching to instantaneous network jitter in the fault-tolerant scheduling method described in this embodiment. This significantly improves the success rate of scheduling the target model while also reducing timeout latency caused by scheduling anomalies.

[0026] Optionally, in one implementation of this embodiment, the method further includes the following steps: Figure 2 As shown, Figure 2 A schematic diagram of a retry waiting time interval control process provided in this application embodiment: Step S21: When the execution process is mapped to the same model retry path, an exponential backoff algorithm with random perturbation is used to control the waiting interval for each retry; wherein, the exponential backoff algorithm calculates the exponential delay base time by exponentially increasing the preset base waiting time according to the current number of retry attempts. Step S22: Add random jitter time generated based on a preset fluctuation range to the index delay reference time to calculate the current retry waiting interval; Step S23: After delaying the retry waiting interval, initiate a scheduling request for the remote large language model service again to perform phase scattering on the physical time axis for concurrent retry requests.

[0027] In the practical application scenario of this embodiment, although exponential backoff is an effective way to control retry intervals, in a large-scale distributed agent cluster, if a massive number of concurrent agents experience scheduling request errors due to network congestion during the same time period, static exponential retries may cause these scheduling requests to retry in phase at the next time point, also known as the thundering herd effect. This could trigger another traffic surge and overwhelm the large model service gateway at the next moment. To avoid this situation, this embodiment limits the scheduling engine to performing the following process steps for each retry waiting time interval within the same model retry path. Step a: Calculate the exponential delay baseline time: Extract the current number of retries N, calculate the exponential growth of the preset base waiting time and the exponential base (e.g., the base waiting time is set to 1.0 seconds and the backoff base is set to 2) at the Nth order, and obtain the current exponential delay baseline time; Step b: Calculate the current retry waiting interval: Generate a random jitter time with positive and negative polarities within a preset fluctuation range (e.g., ±0.5 seconds) (e.g., δ=-0.342 seconds), add it to the baseline value to calculate the current retry waiting interval as t=7.658 seconds; Step c, Delayed Scheduling: The scheduling engine suspends the current task for t seconds, and then re-initiates the request to the remote large model, thus dispersing concurrent traffic on the timeline.

[0028] By implementing the above steps, this embodiment can completely flatten the instantaneous traffic spikes caused by massive concurrent retry traffic, making the retry peaks uniformly and smoothly stretched, thereby greatly alleviating the concurrent pressure of the remote large model and significantly improving the self-healing success rate of the same model under congestion faults.

[0029] Optionally, in one implementation of this embodiment, the step of retrieving the first candidate model from a preset global candidate model set that does not belong to the set of failed models for scheduling switching includes: using the set of failed models as an exclusion set, filtering the global candidate model list, removing models that have failed during the current agent scheduling task execution period, and obtaining a subset of candidate available models; selecting the first model from the subset of candidate available models as the current candidate model according to the priority order of the global candidate model list; and redirecting the routing pointer of the agent scheduling task to the candidate model to achieve scheduling switching.

[0030] To ensure that during model switching, a backup model suitable for seamless task execution can be retrieved in the fastest and most efficient way, the system extracts the set of failed models as an exclusion set blacklist when a model switching decision is triggered. This blacklist filters the global candidate model list, excluding all failed models that have occurred during the task's execution lifecycle, generating a "subset of candidate usable models." Then, based on the priority of the global candidate list, the system retrieves and selects the top-ranked healthy model as the current candidate model, redirects the agent task's scheduling route pointer to that model, and initiates a new scheduling process. This completely avoids repetitive and meaningless trial-and-error collisions involving incompatible large model nodes for the same specific task, further reducing the probing and jitter time of alternative paths.

[0031] Optionally, in one implementation of this embodiment, the method further includes: associating and recording the timestamp of the failure time corresponding to the scheduling failure of each fault model in the failure model set; Before switching the first candidate model that does not belong to the set of failed models from the preset global candidate models, the current system scheduling time is obtained, and the elapsed time between the system scheduling time and the timestamp of the failure time corresponding to each fault model in the set of failed models is calculated. When the elapsed duration corresponding to any fault model exceeds the preset reset decay duration threshold, the fault model is removed from the current set of failed models and its eligibility to be retrieved is restored in the global candidate model list.

[0032] In the practical application scenarios of this application embodiment, if a large model that has experienced a scheduling failure is permanently frozen in the set of failed models without any changes, in a big data analysis intelligent agent service system operating on a long-term scale, it is easy for the main large model to be blindly excluded even after the environment recovers, causing a significant long-term performance degradation of the inference service. To avoid this phenomenon, this embodiment introduces an elastic regression scoring mechanism with a temporal physical fading mechanism in this step. Specifically, the set of failed models is converted into key-value pairs that record the faulty model identifier and the timestamp of the failure time. Before switching the decision to retrieve the next candidate model, the scheduling engine can execute the following dynamic recovery control strategy, such as... Figure 3 As shown, Figure 3 A flowchart illustrating a dynamic recovery control strategy provided in this application embodiment includes: Step S31: Extract the system physical timestamp (t_{current}) at the current decision moment; Step S32: Calculate the difference between the current system time and the failure time timestamp (t_{fail}) corresponding to each failure model recorded in the failed model set to obtain the actual physical elapsed time; Step S33: If it is determined that the physical elapsed time is greater than the preset reset decay time threshold (e.g., a half-decay period of 30 minutes is set), it means that the temporary fault of the previously scheduled fault model has been repaired. Then, a dynamic recovery action is performed to remove the fault model from the blacklist exclusion set (the set of failed models) and restore its eligibility to be searched and scheduled.

[0033] By implementing the above steps, the method described in this embodiment effectively breaks the information deadlock and availability blockage caused by static blacklists. While ensuring rapid avoidance of short-term errors, it enables the optimal large-scale inference model to automatically recover schedulability over long-term scales, thereby maintaining a good balance between system resource utilization and inference performance over long periods.

[0034] Optionally, in one implementation of this embodiment, the method further includes: synchronously storing the set of failed models as part of the metadata of the current agent scheduling task into the system's non-volatile memory; When the scheduling engine process running the agent scheduling task crashes and restarts abnormally, the set of failed models, the number of retries, and the task scheduling context before the agent scheduling task crash are read from and deserialized from the non-volatile memory to restore and continue the original fault-tolerant scheduling process.

[0035] In the practical application scenario described above, namely the high-latency scheduling environment of large models, the scheduling process or server may crash the executing agent service at any time due to power failure, restart, or other reasons, losing historical exclusion memory and causing repeated collisions after restart. To avoid this situation, this embodiment converts the failed model set array, timestamp metadata, number of retries, and other task scheduling status information into JSON format in real time and writes it synchronously to non-volatile storage (such as external non-volatile Redis or distributed storage) with each service update. When a process crashes and restarts, the crash scene data is reread from this non-volatile storage and deserialized to restore the crash scene data, thereby continuing the original scheduling inference process sequentially at the crash node. In this way, high availability is guaranteed for the system's fault-tolerant decision-making, reducing the probability of repeated collision quotas and resource waste after crash recovery.

[0036] Optionally, in one implementation of this embodiment, the step of converting the current state of the periodic task to a reset state so as to reschedule the running agent to execute the periodic task when the next trigger cycle arrives includes the following steps, such as... Figure 4 As shown, Figure 4 A flowchart illustrating an execution cycle scheduling task provided in an embodiment of this application is as follows: Step S41: Configure a periodic task state machine to manage the state of the periodic task, which includes an idle state, a running state, a retry state, and a reset state. Step S42: When a timed scheduling event occurs, the periodic task is switched from the idle state to the running state to start the scheduling of the current triggering cycle; Step S43: When scheduling fails and the number of retries has not reached the maximum retry threshold, the periodic task is switched to the retry state to perform a retry with the same model or a model switch. Step S44: When the scheduling retry is successful, the periodic task is transferred from the retry state back to the idle state, and the set of failed models and the current error state of the task are reset. Step S45: When the number of retries reaches the maximum retry threshold, the periodic task is switched from the retry state to the reset state. In the reset state, the failed model set is cleared, the number of retries and the current error state of the task are reset, and then the task is switched to the idle state to suspend and wait for the next trigger cycle.

[0037] In this implementation, to provide a more rigorous state control model for the timed recovery of periodic tasks and prevent task scheduling chaos or state loss due to various race conditions, this embodiment configures a four-state machine (Idle, Running, Retry, Reset) in the background through the above steps: When a timed event is triggered, the state transitions from Idle to Running; when a scheduling failure occurs and the number of retries has not exceeded the limit, the flow transitions to Retry, executing a backoff retrieval of the same model or model switching, and accumulating the retry counter in real time; if the retry is successful, Retry returns to Idle and resets the error state and the set of failed models; if the count reaches the maximum retry threshold, it transitions to the Reset state. In the Reset state, the set of failed models, the counter, and the error state are cleared, and after resetting, it returns to Idle, waiting for the next trigger. This establishes a rigorous and secure closed-loop logic for periodic state transitions, ensuring the timing and execution determinism of the fault-tolerant scheduling process under long-term unattended operation.

[0038] Optionally, in one implementation of this embodiment, the method further includes: when in a reset state, recording in the audit log a structured skip event that skips the current triggering period due to service scheduling failure when executing the periodic task; if, when executing the periodic task, it is detected that the structured skip event occurs in multiple consecutive triggering periods of a preset number, then triggering a long-term service failure alarm for the periodic task.

[0039] In the practical application scenario of this embodiment, the periodic self-healing process can greatly reduce the alarm noise of short-term, occasional jitter. However, if a large model experiences a persistent paralysis lasting for several days due to line damage, and the scheduling system continues to repeatedly execute the self-healing reset process, it will mask system-level major failures, causing a loss of system observability. Therefore, when the state machine transitions to the Reset state, a structured log containing the skip_event of skipping the current period can be written to the background audit database. An independent external alarm monitoring system (such as Prometheus) subscribes to this audit log stream and establishes a sliding statistical window for the number of skip_events: if the task is detected to have occurred multiple times consecutively within the sliding window (e.g., 3 times, representing 3 consecutive periods of self-healing reset and skipping), it is determined that the external service is completely paralyzed, the alarm is automatically escalated, and a work order is dispatched manually. This effectively solves the traditional defect of "self-healing systems selectively blinding to long-term hard obstacles," allowing short-term, occasional failures to recover silently within the state machine, while ensuring that long-term failures can be keenly detected and identified.

[0040] Optionally, in one implementation of this embodiment, the method further includes: during the fault-tolerant scheduling process, acquiring at least one of the following information: error classification matching log, mapping decision record, candidate model selection action during model switching, and periodic task state machine transition event corresponding to each scheduling failure; encapsulating the acquired information into structured audit data containing occurrence timestamp, task identifier, failure model, error classification category, and corresponding execution processing action, and persistently writing it into a distributed audit log system.

[0041] In the practical application scenario of the above steps in this embodiment, scheduling large model agents requires expensive API quota billing costs. Therefore, it is necessary to provide a usable, complete, and highly observable audit trail for this scheduling process to facilitate later tracing, review, and billing analysis. Therefore, this step in this embodiment is limited to asynchronously capturing current event information when the scheduling state changes (such as classification matching, path routing mapping, appending failed model sets, or periodic skipping). This event information includes a timestamp, scheduling task ID, involved faulty models, error categories, and JSON audit data of decision-making actions. This data is then persistently written to a distributed auditing system such as ClickHouse or Elasticsearch for storage. This provides a complete data source for parsing and replaying the scheduling trajectory, thus providing intuitive data guidance for improving the long-term stability of large model agent services and optimizing system service costs.

[0042] The fault-tolerant scheduling method for model services provided in this application extracts failure feature data from acquired model scheduling failure information; uses a preset error classifier to determine the error category corresponding to the scheduling failure information based on the failure feature data, where the error category includes at least one of transient error type and persistent error type; according to the error category, maps the execution flow of the current agent scheduling task for the remote large language model to a same model retry path or a model switching path; if the execution flow is mapped to the model switching path, it reads the set of failed models recorded in the metadata of the agent scheduling task, retrieves the first candidate model that does not belong to the set of failed models from the preset global candidate models, performs the scheduling switch, and adds the remote large language model before the switch as the fault model to the set of failed models. This method uses an error classifier to match the failure feature data extracted from the scheduling failure information to accurately separate transient and persistent errors. For transient errors, priority is given to mapping to the same model's retry path for backoff, preserving the business advantages of the preferred large model in terms of response quality and performance to the greatest extent, and significantly reducing switching costs; for persistent errors, direct route switching is performed to avoid meaningless retries, greatly saving billing costs and waiting processing latency. It also effectively ensures that routing does not get stuck in an infinite loop.

[0043] Example 2 Based on the fault-tolerant scheduling method for model services provided in Embodiment 1 of this application, this embodiment also provides a corresponding fault-tolerant scheduling system for model services, such as... Figure 5 As shown, Figure 5 This application provides a schematic diagram of the structure of a fault-tolerant scheduling system 50 for a model service, which includes: The acquisition module 501 is used to acquire scheduling failure information and extract failure feature data from the scheduling failure information; the scheduling failure information is generated when the remote large language model service fails to schedule. The classification module 502 is used to determine the error category corresponding to the scheduling failure information based on the failure feature data using a preset error classifier, wherein the error category includes at least one of transient error type and persistent error type; The mapping module 503 maps the execution flow of the current agent scheduling task for the remote large language model to the same model retry path or model switching path according to the error category. The switching module 504 is used to read the set of failed models recorded in the metadata of the agent scheduling task if the execution process is mapped to the model switching path, retrieve the first candidate model that does not belong to the set of failed models from the preset global candidate models, switch the model, and add the remote large language model before the switch as the fault model to the set of failed models.

[0044] Optionally, in one implementation of this embodiment, the system 50 further includes a delay module (not shown in the figures), which is used to accumulate the number of retries of the agent scheduling task; when the agent scheduling task is a periodic task triggered at a time, and the accumulated number of retries reaches a preset maximum retry threshold, the current state of the periodic task is converted to a reset state so that the agent can be rescheduled to execute the periodic task when the next triggering cycle arrives.

[0045] Optionally, in one implementation of this embodiment, the acquisition module 501 is further configured to intercept the large language model interface request at the network transport layer or the large language model interface middleware layer, and acquire the heterogeneous original error data packet or underlying connection abnormal error information returned by the remote large language model service scheduling failure, as the scheduling failure information. The corresponding Hypertext Transfer Protocol status code, error message text, and original error type field defined by the large model vendor are parsed and extracted from the scheduling failure information as the failure feature data.

[0046] Optionally, in one implementation of this embodiment, the classification module 502 is further configured to: configure the preset error classifier to perform semantic analysis on the failure feature data based on a machine learning text classification model, or to use a large language model as a classifier, so as to perform multi-dimensional feature recognition on the scheduling failure information and determine the error category corresponding to the scheduling failure information.

[0047] Optionally, in one implementation of this embodiment, the classification module 502 is further configured to: match the error message text in the failure feature data with a preset set of regular features to obtain a matching result; and based on the matching result, determine the error category of the scheduling failure, wherein the category includes at least one of the following: quota exhaustion category, rate limiting category, model overload category, network timeout category, identity authentication error category, and general error category. Among them, the quota exhaustion category and the identity authentication error category belong to the persistent error type, while the rate limiting category, the model overload category, and the network timeout category belong to the transient error type.

[0048] Optionally, in one implementation of this embodiment, the mapping module 503 is further configured to: when the error category is the quota exhaustion category or the model overload category, map the current execution flow to the model switching path; When the current error category is the rate limiting category or the network timeout category, the current execution flow will be mapped to the same model retry path; When the current error category is the identity authentication error category, a preset alarm policy is triggered and the current agent scheduling task is suspended to wait for external operation and maintenance intervention. When the current error category is the general error category, the current execution flow is mapped to the same model retry path, and a conservative retry decision with a limited number of retries is made.

[0049] Optionally, in one implementation of this embodiment, the system 50 further includes an adjustment module (not shown in the figures). This adjustment module is used to control the retry waiting interval for each retry by employing an exponential backoff algorithm with random perturbation when the execution flow is mapped to a retry path of the same model. The exponential backoff algorithm calculates an exponential delay base time by exponentially increasing a preset base waiting time based on the current number of retry attempts. A random jitter time generated based on a preset fluctuation range is superimposed on the exponential delay base time to calculate the current retry waiting interval. After delaying the retry waiting interval, a scheduling request for the remote large language model service is initiated again to perform phase scattering on the physical time axis for concurrent retry requests.

[0050] Optionally, in one implementation of this embodiment, the switching module 504 is further configured to: use the set of failed models as an exclusion set, filter the global candidate model list, remove models that have failed during the current agent scheduling task execution period, and obtain a subset of candidate available models; According to the priority order of the global candidate model list, the first model in the subset of available candidate models is selected as the current candidate model; The routing pointer of the intelligent agent's scheduling task is redirected to the candidate model to achieve scheduling switching.

[0051] Optionally, in one implementation of this embodiment, the system 50 further includes an association module (not shown in the figures), used to associate and record the failure time timestamp corresponding to the scheduling failure of each fault model in the failure model set; and before retrieving the first candidate model that does not belong to the set of scheduled failure models from the preset global candidate models for scheduling switching, to obtain the current system scheduling time, and to calculate the elapsed time between the system scheduling time and the failure time timestamps corresponding to each fault model in the set of failed models; When the elapsed duration corresponding to any fault model exceeds the preset reset decay duration threshold, the fault model is removed from the current set of failed models and its eligibility to be retrieved is restored in the global candidate model list.

[0052] Optionally, in one implementation of this embodiment, the system 50 further includes a first storage module (not shown in the figures), used to synchronously store the failed model set as part of the metadata of the current agent scheduling task in the system's non-volatile memory; when the scheduling engine process running the agent scheduling task crashes and restarts abnormally, the failed model set, the number of retries, and the task scheduling context before the agent scheduling task crash are read from the non-volatile memory and deserialized to restore the original fault-tolerant scheduling process.

[0053] Optionally, in one implementation of this embodiment, the delay module is further configured to configure a periodic task state machine to manage the state of the periodic task, which includes an idle state, a running state, a retry state, and a reset state. When a timed scheduling event occurs, the periodic task is switched from the idle state to the running state to start the scheduling of the current triggering cycle; When scheduling fails and the number of retries has not reached the maximum retry threshold, the periodic task is switched to the retry state to perform a retry of the same model or a model switch. When the scheduling retry is successful, the periodic task is transferred from the retry state back to the idle state, and the set of failed models and the current error state of the task are reset. When the number of retries reaches the maximum retry threshold, the periodic task is switched from the retry state to the reset state. In the reset state, the failed model set is cleared, the number of retries and the current error state of the task are reset, and then the task is switched to the idle state to suspend and wait for the next trigger cycle.

[0054] Optionally, in one implementation of this embodiment, the system 50 further includes a warning module (not shown in the figures), which is used to record in the audit log, when in a reset state, a structured skip event that skips the current triggering period due to service scheduling failure when executing the periodic task; if the structured skip event is detected to occur in a preset number of consecutive triggering periods when executing the periodic task, a long-term service failure alarm for the periodic task is triggered.

[0055] Optionally, in one implementation of this embodiment, the system 50 further includes a second storage module (not shown in the figures), which is used to acquire at least one of the following information during the fault-tolerant scheduling process: error classification matching log, mapping decision record, candidate model selection action during model switching, and periodic task state machine transition event. The information obtained above is encapsulated into structured audit data containing timestamps, task identifiers, failure models, error classifications, and corresponding execution actions, and persistently written to the distributed audit log system.

[0056] The fault-tolerant scheduling system for model services provided in this application includes an acquisition module that extracts failure feature data from acquired model scheduling failure information; a classification module that uses a preset error classifier to determine the error category corresponding to the scheduling failure information based on the failure feature data, wherein the error category includes at least one of transient error type and persistent error type; a mapping module that maps the execution flow of the current agent scheduling task for the remote large language model to a retry path or a model switching path according to the error category; and a switching module that, if the execution flow is mapped to the model switching path, reads the set of failed models recorded in the metadata of the agent scheduling task, retrieves the first candidate model that does not belong to the set of failed models from the preset global candidate models, performs the scheduling switch, and adds the remote large language model before the switch as the fault model to the set of failed models. This method uses an error classifier to match the failure feature data extracted from the scheduling failure information to accurately separate transient and persistent errors. For transient errors, priority is given to mapping to the same model's retry path for backoff, preserving the business advantages of the preferred large model in terms of response quality and performance to the greatest extent, and significantly reducing switching costs; for persistent errors, direct route switching is performed to avoid meaningless retries, greatly saving billing costs and waiting processing latency. It also effectively ensures that routing does not get stuck in an infinite loop.

[0057] Example 3 This application also provides a storage medium storing a computer program that, when executed by a processor, implements a fault-tolerant scheduling method for any of the model services described in the foregoing embodiment one of this application.

[0058] Example 4 This application also provides an electronic device, such as... Figure 6 As shown, Figure 6 This application provides a schematic diagram of the structure of an electronic device 30, which includes: One or more processors 301, communication interface 302, memory 303 and communication bus 304, the processors 301, memory 303 and communication interface 302 communicate with each other through communication bus 304; Memory 303 is used to store one or more programs; When the one or more programs are executed by the one or more processors 301, the one or more processors 301 implement any of the fault-tolerant scheduling methods for model services as described in Embodiment 1 of this application.

[0059] This application has now described specific embodiments of the subject matter. In some cases, the actions described in the claims can be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing can be advantageous.

[0060] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to methodologies). However, with technological advancements, many methodological improvements today can be considered direct improvements to hardware circuit structures. Information gatherers almost always obtain the corresponding hardware circuit structure by programming the improved methodologies into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Information gatherers can program and "integrate" a digital system layer onto a PLD themselves, without needing chip manufacturers to gather information and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0061] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0062] The system layers, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0063] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0064] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0065] Those skilled in the art will understand that embodiments of this application can be provided as methods, system-level, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0066] This application can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific transactions or implement specific abstract data types. This application can also be practiced in distributed computing environments where transactions are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0067] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system-level embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0068] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A fault-tolerant scheduling method for model services, characterized in that, include: Obtain scheduling failure information and extract failure feature data from the scheduling failure information; The scheduling failure information is generated when the remote large language model service fails to schedule. Using a preset error classifier based on the failure feature data, the error category corresponding to the scheduling failure information is determined, and the error category includes at least one of transient error type and persistent error type; Based on the error category, the execution flow of the current agent scheduling task for the remote large language model is mapped to the same model retry path or the model switching path. If the execution process is mapped to the model switching path, the set of failed models recorded in the metadata of the agent scheduling task is read, the first candidate model that does not belong to the set of failed models is retrieved from the preset global candidate models and the model is switched, and the remote large language model before the switch is added to the set of failed models as a fault model.

2. The method according to claim 1, characterized in that, The method further includes: accumulating the number of retries for the agent's scheduled task; When the agent scheduling task is a periodic task triggered at a time, and the accumulated number of retries reaches the preset maximum retry threshold, the current state of the periodic task is changed to a reset state so that the agent can be rescheduled to execute the periodic task when the next triggering cycle arrives.

3. The method according to claim 1, characterized in that, The step of obtaining scheduling failure information and extracting failure feature data from the scheduling failure information includes: Intercept the large language model interface request at the network transport layer or the large language model interface middleware layer, and obtain the heterogeneous original error data packet or underlying connection exception error information returned by the remote large language model service scheduling failure as the scheduling failure information; The corresponding Hypertext Transfer Protocol status code, error message text, and original error type field defined by the large model vendor are parsed and extracted from the scheduling failure information as the failure feature data.

4. The method according to claim 1 or 2, characterized in that, The step of using a preset error classifier to determine the error category corresponding to the scheduling failure information based on the failure feature data includes: The preset error classifier is configured to perform semantic analysis on the failure feature data based on a machine learning text classification model or a large language model as a classifier, so as to perform multi-dimensional feature recognition on the scheduling failure information and determine the error category corresponding to the scheduling failure information.

5. The method according to claim 1 or 2, characterized in that, The step of using a preset error classifier to determine the error category corresponding to the scheduling failure information based on the failure feature data includes: The error message text in the failure feature data is matched with a preset set of regular features to obtain the matching result; Based on the matching results, the error category of scheduling failure is determined. The category includes at least one of the following: quota exhaustion category, rate limiting category, model overload category, network timeout category, identity authentication error category, and general error category. Among them, the quota exhaustion category and the identity authentication error category belong to the persistent error type, while the rate limiting category, the model overload category, and the network timeout category belong to the transient error type.

6. The method according to claim 5, characterized in that, The step of mapping the execution flow of the current agent scheduling task for the remote large language model to a retry path or a model switching path based on the error category includes: When the error category is the quota exhaustion category or the model overload category, the current execution flow will be mapped to the model switching path; When the current error category is the rate limiting category or the network timeout category, the current execution flow will be mapped to the same model retry path; When the current error category is the identity authentication error category, a preset alarm policy is triggered and the current agent scheduling task is suspended to wait for external operation and maintenance intervention. When the current error category is the general error category, the current execution flow is mapped to the same model retry path, and a conservative retry decision with a limited number of retries is made.

7. The method according to claim 1, characterized in that, The method further includes: when the execution flow is mapped to a retry path of the same model, an exponential backoff algorithm with random perturbation is used to control the waiting interval for each retry; The exponential backoff algorithm calculates the exponential delay baseline time by exponentially increasing the preset base waiting time based on the current number of retry attempts. The current retry waiting interval is calculated by superimposing the random jitter time generated based on the preset fluctuation range into the index delay base time; After delaying the retry waiting interval, a scheduling request for the remote large language model service is initiated again to perform phase scattering on the physical time axis for concurrent retry requests.

8. The method according to claim 1, characterized in that, The step of retrieving the first candidate model from the preset global candidate models that does not belong to the set of failed scheduling models and switching the scheduling includes: The set of failed models is used as an exclusion set to filter the global candidate model list, removing models that have failed during the current agent scheduling task execution period, thus obtaining a subset of candidate usable models; According to the priority order of the global candidate model list, the first model in the subset of available candidate models is selected as the current candidate model; The routing pointer of the intelligent agent's scheduling task is redirected to the candidate model to achieve scheduling switching.

9. The method according to claim 1, characterized in that, The method further includes: Associate and record the timestamp of the failure moment corresponding to the scheduling failure of each fault model in the failure model set; Before switching the first candidate model that does not belong to the set of failed models from the preset global candidate models, the current system scheduling time is obtained, and the elapsed time between the system scheduling time and the timestamp of the failure time corresponding to each fault model in the set of failed models is calculated. When the elapsed duration corresponding to any fault model exceeds the preset reset decay duration threshold, the fault model is removed from the current set of failed models and its eligibility to be retrieved is restored in the global candidate model list.

10. The method according to claim 1, characterized in that, The method further includes: synchronously storing the set of failed models as part of the metadata of the current agent scheduling task into the system's non-volatile memory; When the scheduling engine process running the agent scheduling task crashes and restarts abnormally, the set of failed models, the number of retries, and the task scheduling context before the agent scheduling task crash are read from and deserialized from the non-volatile memory to restore and continue the original fault-tolerant scheduling process.

11. The method according to claim 2, characterized in that, The step of converting the current state of the periodic task to a reset state, so as to reschedule the running agent to execute the periodic task when the next trigger cycle arrives, includes: Configure a periodic task state machine to manage the state of the periodic task, which includes idle state, running state, retry state and reset state; When a timed scheduling event occurs, the periodic task is switched from the idle state to the running state to start the scheduling of the current triggering cycle; When scheduling fails and the number of retries has not reached the maximum retry threshold, the periodic task is switched to the retry state to perform a retry of the same model or a model switch. When the scheduling retry is successful, the periodic task is transferred from the retry state back to the idle state, and the set of failed models and the current error state of the task are reset. When the number of retries reaches the maximum retry threshold, the periodic task is switched from the retry state to the reset state. In the reset state, the failed model set is cleared, the number of retries and the current error state of the task are reset, and then the task is switched to the idle state to suspend and wait for the next trigger cycle.

12. The method according to claim 11, characterized in that, The method further includes: When in a reset state, a structured skip event is recorded in the audit log to identify when the current triggering cycle is skipped due to service scheduling failure during the execution of the periodic task. If, during the execution of the periodic task, it is detected that the structured skip event occurs within a preset number of consecutive triggering cycles, a long-term service failure alarm for the periodic task is triggered.

13. The method according to claim 2, characterized in that, The method further includes: During the fault-tolerant scheduling process, at least one of the following information is obtained: error classification matching log, mapping decision record, candidate model selection action during model switching, and periodic task state machine transition event corresponding to each scheduling failure; The information obtained above is encapsulated into structured audit data containing timestamps, task identifiers, failure models, error classifications, and corresponding execution actions, and persistently written to the distributed audit log system.

14. A fault-tolerant scheduling system for model services, characterized in that, include: The acquisition module is used to acquire scheduling failure information and extract failure feature data from the scheduling failure information. The scheduling failure information is generated when the remote large language model service fails to schedule. The classification module is used to determine the error category corresponding to the scheduling failure information based on the failure feature data using a preset error classifier. The error category includes at least one of transient error type and persistent error type. The mapping module maps the execution flow of the current agent scheduling task for the remote large language model to the same model retry path or model switching path according to the error category. The switching module is used to read the set of failed models recorded in the metadata of the agent scheduling task if the execution process is mapped to the model switching path, retrieve the first candidate model that does not belong to the set of failed models from the preset global candidate models, switch the model, and add the remote large language model before the switch as the fault model to the set of failed models.

Images