An agent task request scheduling method, an electronic device, and a storage medium

Abstract

The present disclosure provides a task request scheduling method of an agent, an electronic device and a storage medium. The method comprises: receiving a task request to be executed and parsing a task attribute of the task request; determining a priority of the task request according to the task attribute based on a preset configuration mapping table; wherein the configuration mapping table stores a mapping relationship between a task attribute and a priority; distributing the task request to a priority queue; reading the task request in the priority queue in order from high to low according to the priority, and calling an agent to execute the task request read from the priority queue.

Description

Technical Field

[0001] This disclosure relates to the field of task scheduling technology, and more specifically, to a task request scheduling method for an intelligent agent, an electronic device, and a computer-readable storage medium. Background Technology

[0002] With the rapid development of Large Language Model (LLM) technology, agent-based application scenarios are becoming increasingly diverse, encompassing various types such as real-time dialogue, deep research report generation, text-based video, image-based video, and artificial intelligence (AI) writing. These scenarios exhibit significant differences in task scheduling requirements: real-time scenarios have high requirements for response latency, while long-duration scenarios allow for longer processing cycles. Given the limited computing resources such as Graphics Processing Units (GPUs), ensuring the success rate of user request processing and maintaining cluster stability have become key technical challenges that need to be addressed.

[0003] In related technologies, separate scheduling systems are typically built for real-time and long-duration scenarios, making it difficult to achieve unified access protocols, queue management, and rate limiting control. Furthermore, these solutions generally lack multi-dimensional, fine-grained scheduling capabilities based on user attributes; adjusting task priorities often relies on code modifications or service restarts, failing to quickly respond to sudden task demands. Therefore, a task scheduling solution is needed that can uniformly support multiple task scenarios, enable differentiated scheduling based on user attributes, and possess runtime configurability. Summary of the Invention

[0004] One objective of this disclosure is to provide a technical solution for scheduling task requests of an intelligent agent, so as to achieve fine differentiation of task value and differentiated allocation of computing resources.

[0005] According to a first aspect of this disclosure, a task request scheduling method for an intelligent agent is provided, the method comprising: Receive a task request to be executed and parse the task attributes of the task request; Based on a preset configuration mapping table, the priority of the task request is determined according to the task attributes; wherein, the configuration mapping table stores the mapping relationship between task attributes and priorities; Distribute the task requests carrying the corresponding priorities to the priority queue; The task requests in the priority queue are read in descending order of priority, and the agent is invoked to execute the task requests read from the priority queue.

[0006] Optionally, the task attributes include any one or more of the following: region, user type, membership level, request time period, and task type.

[0007] Optionally, the step of reading task requests in the priority queue in descending order of priority and invoking an agent to execute the task requests read from the priority queue includes: when reading the task requests in the priority queue in descending order of priority, performing distributed rate limiting verification on the task requests read from the priority queue; and invoking the agent to execute the verified task requests.

[0008] Optionally, the task attribute includes a task type; the distributed rate limiting verification of task requests read from the priority queue includes: performing query rate limiting verification per second on task requests read from the priority queue based on the token bucket algorithm according to the task type; and / or performing concurrency rate limiting verification on task requests read from the priority queue based on the semaphore mechanism.

[0009] Optionally, the method further includes: detecting the backlog length of existing task requests in the first-in-first-out queue; when the backlog length exceeds a preset length threshold, distributing newly received task requests to the priority sorting queue; reading existing task requests in the first-in-first-out queue, calculating the priority of the existing task requests according to the configuration mapping table, and distributing the existing task requests to the priority sorting queue.

[0010] Optionally, the method further includes: detecting the queuing time of the task request in the priority queue; detecting whether the task request meets the discarding condition based on the task attribute, the queuing time, and the priority; if the task request meets the discarding condition, removing the task request from the priority queue and returning a discard response to the sender of the task request.

[0011] Optionally, the task attribute further includes a task type, and the step of distributing the task request carrying the corresponding priority to the priority queue includes: determining a target storage medium that matches the task type; and distributing the task request carrying the corresponding priority to the priority queue corresponding to the target storage medium.

[0012] Optionally, the method further includes: receiving an update instruction for the configuration mapping table through a configuration center, and performing a hot update on the configuration mapping table.

[0013] Optionally, the task attribute further includes a task type, and the method further includes: obtaining an event routing relationship pre-configured for the task type; and updating the task status of the task request according to the event routing relationship.

[0014] According to a second aspect of the present disclosure, an electronic device is also provided, including a memory and a processor, the memory being used to store a computer program; the processor being used to execute the computer program to implement the method according to the first aspect of the present disclosure.

[0015] According to a third aspect of the present disclosure, a computer-readable storage medium is also provided, on which a computer program is stored, the computer program implementing the method according to the first aspect of the present disclosure when executed by a processor.

[0016] This embodiment first parses the task attributes of the task request, then automatically maps and generates corresponding priorities based on a configurable configuration mapping table, distributes the task requests to a priority queue, and executes them in an orderly manner according to their own priorities. This closed-loop scheduling logic abandons the traditional coarse scheduling mode of uniform FIFO scheduling of task requests, which cannot distinguish between task requests and task attributes. It achieves fine differentiation of task value and differentiated allocation of computing resources, effectively avoiding the problem of high-value users and core task requests being squeezed and blocked by ordinary low-priority task requests in peak traffic scenarios. It significantly reduces the queuing latency of high-priority task requests and improves the execution success rate of core task requests. At the same time, the scheduling strategy can be dynamically configured based on configurable mapping rules, without the need for hard-coding to adapt to scenario iterations. It takes into account the flexibility and accuracy of task request scheduling and the stability of the task request scheduling system. While ensuring the overall task request throughput capacity, it maximizes the utilization value of GPU computing resources and the user experience of core task requests.

[0017] Other features and advantages of the embodiments of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the disclosure.

[0019] Figure 1 This is a schematic diagram of the architecture of a task request scheduling system for an intelligent agent that can apply the methods of the embodiments of this disclosure; Figure 2 This is a flowchart illustrating a task request scheduling method for intelligent agents according to some embodiments.

[0020] Figure 3This is a schematic diagram of the event routing relationship of the DRS task request for the college entrance examination according to some embodiments.

[0021] Figure 4 This is a schematic diagram of the event routing relationship of video generation task requests according to some embodiments.

[0022] Figure 5 This is a block schematic diagram of an electronic device according to one embodiment. Detailed Implementation

[0023] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0024] The following description of at least one exemplary embodiment is merely illustrative and is not intended to limit the scope of this disclosure or its application or use.

[0025] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0026] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0027] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0028] It should be noted that all data acquisition actions in this disclosure were carried out in compliance with the relevant data protection laws and policies of the country where the data is located, and with the authorization of the relevant equipment owner.

[0029] This disclosure relates to a task request scheduling scheme for an intelligent agent. Figure 1 This is a schematic diagram of the architecture of a task request scheduling system for an intelligent agent that can apply the methods of the embodiments of this disclosure. For example... Figure 1 As shown, the task request scheduling system 100 adopts a four-layer architecture design, including an access layer 110, a task layer 120, an execution layer 130, and a storage layer 140, which performs unified management of task queuing, rate limiting, and retry between the task system and the intelligent agent.

[0030] The access layer 110 supports agent access in both real-time and long-duration scenarios, providing a unified access protocol. The task layer 120 is a task extension layer, enabling configurable access based on different agent protocols and retry strategies. The execution layer 130 is the core logic layer, containing core capabilities such as task orchestration logic, task state management, queue management, and rate limiting. The storage layer 140 provides a unified abstraction for message storage and key-value (KV) storage. Message storage handles task queue queuing, while KV storage saves state information during task execution, agent return results, and checkpoint information.

[0031] exist Figure 1 In the task request scheduling system shown, when the concurrent traffic of large language model (LLM) application scenarios (including real-time chatbox, deep research report DRS, text-based video, image-based video, and artificial intelligence (AI) writing) increases, one possible implementation is to build separate task request scheduling systems for real-time and long-duration scenarios, and use traditional first-in-first-out message queues for task request queuing and distribution. However, this approach lacks fine-grained scheduling capabilities based on task attributes, and priority adjustments depend on code modifications or service restarts, resulting in problems such as scenario fragmentation, coarse scheduling, and delayed response.

[0032] To address this issue, this embodiment introduces a preset configuration mapping table, calculates the priority of the task request based on its task attributes, distributes the requests to a priority sorting queue, and calls the intelligent agent to execute the task in descending order of priority. This ensures the success rate of requests from high-value users under limited computing resources, thus solving the aforementioned technical problem.

[0033] The following combination Figure 1 The present disclosure describes various embodiments of a task request scheduling system.

[0034] <First Embodiment> Figure 2 This is a flowchart illustrating a task request scheduling method for an intelligent agent according to some embodiments. This task request scheduling method for the intelligent agent can be implemented by a task request scheduling system. Specifically, the task request scheduling method for the intelligent agent can be as follows: Figure 1 The task request scheduling system 100 shown is implemented. For example... Figure 2 As shown, the task request scheduling method for the intelligent agent in this embodiment may include the following steps S210 to S240.

[0035] Step S210: Receive the task request to be executed and parse the task attributes of the task request.

[0036] This step involves the access layer uniformly accepting all task requests initiated by various intelligent agents, shielding the differences between different task types and request protocols, and extracting, parsing, and structurally normalizing the original task request data to obtain core scheduling feature data, namely task attributes, for subsequent priority calculation, queue distribution, rate limiting verification, and intelligent discarding.

[0037] Task attributes can be a set of attributes parsed from a task request to identify the source and background of the task, including user attributes describing the user who initiated the task request and / or task type describing the task request itself. Task attributes serve as the core basis for subsequent priority mapping and queue classification and distribution. They are the core data support for realizing refined, differentiated, and configurable intelligent agent scheduling, enabling the task request scheduling system to distinguish between high- and low-value task requests and users, and to achieve optimal allocation of computing resources.

[0038] Specifically, feature data can be parsed and extracted from the task request message, request header, task pass-through parameters, user identity token, request timestamp, and task request identifier. The extracted feature data is then structured and normalized to obtain standardized fields that can be recognized by the configuration mapping table. These standardized fields are then combined to form task attributes.

[0039] In some embodiments, task attributes include any one or more of the following: region, user type, membership level, request time period, and task type.

[0040] The geographical area can be refined to the province or city level, such as province=Shanghai, city=Pudong New Area. User type can distinguish between new and returning users, for example, based on whether the registration duration exceeds a set number of days. Membership levels can include different tiers such as regular members, VIP members, and SVIP members. Request time periods can be differentiated into peak and off-peak periods, such as 9:00-12:00 and 14:00-18:00 as peak periods, and the rest of the time as off-peak periods. Through such multi-dimensional task attributes, high-value task requests can be identified more accurately, providing a basis for subsequent differentiated scheduling.

[0041] Step S220: Based on a preset configuration mapping table, determine the priority of the task request according to the task attributes; wherein, the configuration mapping table stores the mapping relationship between task attributes and priorities.

[0042] The configuration mapping table is a structured rule data table that stores the correspondence between task attributes and priorities, unlike traditional hard-coded fixed priorities and single-dimensional priority configuration methods. This configuration mapping table uniformly carries priority scheduling rules for all scenarios, decoupling scheduling rules from scheduling code. It supports flexible iteration of task requests and rapid optimization of scheduling strategies in scenarios with sudden traffic surges, serving as the fundamental carrier for achieving configurable, dynamically adjustable, and fine-grained scheduling.

[0043] For example, a combination of Shanghai location, existing user type, and SVIP membership level corresponds to priority 1 (highest priority); a combination of Beijing location, new user type, and regular membership level corresponds to priority 8 (lower priority). Priority values ​​can range from 1 to 16, with lower numbers indicating higher priority.

[0044] The configuration mapping table is obtained by pre-completing visual configuration and storage in the configuration center. It allows operators to customize and enter multiple sets of task attribute combinations and priority mapping relationships based on task type, user value, and traffic period. It also supports subsequent online modification, addition, and deletion of rules without modifying the underlying scheduling code.

[0045] Priority is the scheduling priority identifier for task requests. It serves as the quantitative basis for task request enqueuing, queuing, consumption, and resource allocation, and is a static priority specific to each individual task request. Priority is the core basis for distributing task requests to the priority sorting queue, determining the queuing position and consumption order of the task request in the queue. It is the core criterion for subsequent high-priority task requests to be executed first, resource allocation in overloaded scenarios, and low-priority task requests to be discarded as a fallback.

[0046] In some examples, the configuration mapping table stores a one-to-one correspondence between multiple sets of task attribute combinations and priority values. Based on the preset configuration mapping table, the priority of the task request is determined according to the task attributes, including: matching the task attributes of the task request with the rule entries in the configuration mapping table one by one; after a target rule entry is matched, the corresponding priority value is read and determined as the priority of the task request.

[0047] In these examples, by pre-setting a configuration mapping table with multiple attribute combinations and accurately matching each rule to determine the priority of task requests, it is possible to achieve composite priority determination with multi-dimensional attribute linkage, making priority determination more in line with the actual value of tasks and greatly improving the accuracy and rationality of computing resource allocation.

[0048] In other examples, the configuration mapping table can also support a weighted calculation mode. In this mode, the configuration mapping table can record the task attributes and quantified values ​​for each dimension. For example, in membership level, the quantified value for ordinary members is 0.2, for VIP members it is 0.6, and for SVIP members it is 1.0; in user type, the quantified value for new users is 0.3, and for returning users it is 1.0; region can be quantified to values ​​between 0.2 and 1.0 based on the importance of the task request; in task importance, the quantified value for non-core tasks is 0.3, and for core tasks it is 1.0; in time period, the quantified value for off-peak hours is 0.4, and for peak hours it is 1.0.

[0049] After obtaining the quantified values ​​of each dimension of the task request, the combined priority is calculated based on the preset weight coefficients of each dimension's task attributes. The calculation formula is: Combined Priority = Σ (Value of each dimension's tag × Corresponding weight coefficient). The weight coefficients of each dimension's task attributes can be configured according to task requirements. For example, the weight coefficient for region can be configured as 0.3, for user type as 0.2, for membership level as 0.3, for task importance as 0.1, and for time period as 0.1. The sum of the weight coefficients for each dimension is 1.0.

[0050] For example, for a task request with the following characteristics: location Shanghai (quantitative value 1.0), user type old user (quantitative value 1.0), membership level SVIP (quantitative value 1.0), task importance as core task (quantitative value 1.0), and request time period as peak time period (quantitative value 1.0), its combined priority original score = 1.0×0.3+1.0×0.2+1.0×0.3+1.0×0.1+1.0×0.1=1.0.

[0051] For example, for a task request that is located in a low-weight region (quantified value 0.2), is a new user (quantified value 0.3), is a regular member (quantified value 0.2), is a non-core task (quantified value 0.3), and is requested during off-peak hours (quantified value 0.4), its combined priority original score = 0.2×0.3+0.3×0.2+0.2×0.3+0.3×0.1+0.4×0.1=0.25.

[0052] After calculating the original priority score, it can be linearly mapped to the priority range of 1 to 16 to obtain the priority. The higher the original score, the smaller the corresponding priority number (i.e., the higher the priority).

[0053] In these examples, the weighted calculation mode can quantify the contribution of each dimension in a finer way, making the priority differences between different combinations of dimension labels smoother and more continuous, avoiding a large jump in priority due to the absence of a certain dimension label, and is suitable for task scenarios with high requirements for priority fineness.

[0054] In some other examples, the configuration of the mapping table binding time period automatic effect strategy can be implemented. Multiple sets of mapping rules can be preset for peak promotion time periods, daily time periods, and off-peak time periods. The task request scheduling system automatically switches the corresponding rule to calculate the priority according to the time period attribute of the task request, so as to realize adaptive scheduling and control of peak and off-peak traffic.

[0055] Step S230: Distribute the task request carrying the corresponding priority to the priority sorting queue.

[0056] In this embodiment, based on the determined priority of the task requests, all task requests with different scheduling values ​​are uniformly delivered to the priority sorting queue, and the priority carried by the task requests is used as the basis for sorting, queuing and consumption within the queue.

[0057] The priority sorting queue is a centralized task storage queue with internal dynamic sorting capabilities. It uses the priority bound to task requests as the core sorting criterion. After all task requests are stored in the priority sorting queue, the queue automatically rearranges the requests according to their priority from highest to lowest. When the scheduling consumer reads task requests, it directly selects the highest-priority task request in the priority sorting queue for dequeueing and execution, regardless of the order in which the task requests were enqueued. Even if high-value task requests are enqueued later than a large number of low-priority task requests, priority scheduling can still be achieved.

[0058] In some examples, in a priority queue, multiple task requests with the same priority can be sorted in order of their request time from earliest to latest.

[0059] This embodiment achieves orderly queuing and differentiation of high- and low-value task requests through a lightweight mode that integrates priority sorting in task requests. This provides queue support for subsequent orderly consumption based on priority and differentiated allocation of computing resources, and is the core queuing step for achieving lightweight, highly flexible, and refined task scheduling.

[0060] Step S240: Read the task requests in the priority queue in descending order of priority, and call the agent to execute the task requests read from the priority queue.

[0061] Based on the priority queue for queuing task requests, the scheduling consumer strictly follows the priority sorting rules, prioritizes reading high-priority task requests from the priority queue, obtains tasks to be consumed in an orderly manner according to the descending priority queuing order, and actively initiates calls to downstream intelligent agents to complete the execution scheduling of task requests.

[0062] The priority descending order reading mechanism is the core scheduling rule that distinguishes this embodiment from the traditional single queue first-in-first-out consumption, and it is an ordered consumption strategy corresponding to a multi-level queue system.

[0063] The queue-based round-robin reading method is a dedicated reading approach adapted to multi-level priority queue clusters, achieving ordered, complete, and non-preemptive balanced reading across multiple queues. Specifically, it can read task requests in descending order of priority, with higher-priority task requests being read first.

[0064] As can be seen from steps S210 to S240 above, this embodiment first parses the task attributes of the task request, then automatically maps and generates the corresponding priority based on the configurable configuration mapping table, distributes the task request to the priority sorting queue, and executes the task in an orderly manner according to its own priority. This closed-loop scheduling logic abandons the traditional coarse scheduling mode of uniform FIFO scheduling of task requests and inability to distinguish between task requests and task attributes. It realizes the fine differentiation of task value and the differentiated allocation of computing resources, effectively avoiding the problem of high-value users and core task requests being squeezed and blocked by ordinary low-priority task requests in peak traffic scenarios. It significantly reduces the queuing latency of high-priority task requests and improves the execution success rate of core task requests. At the same time, the scheduling strategy can be dynamically configured based on the configurable mapping rules, without the need for hard coding to adapt to scenario iterations. It takes into account the flexibility, accuracy and stability of task request scheduling and the task request scheduling system. While ensuring the overall task request throughput capacity, it maximizes the utilization value of GPU computing resources and the user experience of core task requests.

[0065] In some embodiments, reading task requests in a priority queue in descending order of priority and invoking an agent to execute the task requests read from the priority queue includes: when reading task requests in a priority queue in descending order of priority, performing distributed rate limiting verification on the task requests read from the priority queue; and invoking an agent to execute the verified task requests.

[0066] In this embodiment, distributed rate limiting verification may be triggered after reading a task request from the priority queue and before calling the agent to execute it, to verify and screen the execution eligibility of the task request read from the priority queue.

[0067] Task requests that pass verification are allowed to enter the agent's execution process, while task requests that fail verification are blocked from entering the agent's execution process. Task requests that fail verification can be retained in the priority queue to wait for the next round of reading and retry.

[0068] In these examples, by performing distributed rate limiting verification before task requests are executed, high-priority task requests can be prioritized for scheduling while overload interception of sudden high-priority traffic can be achieved. This ensures the execution priority and experience of core task requests while effectively avoiding the risk of a large number of high-priority task requests breaking through the agent's computing power limit, thus achieving a two-way balance between scheduling priority and stability.

[0069] In some embodiments, distributed rate limiting verification is performed on task requests read from the priority queue, including: performing query rate per second rate limiting verification on task requests read from the priority queue based on the token bucket algorithm according to the task type; and / or performing concurrency rate limiting verification on task requests read from the priority queue based on the semaphore mechanism.

[0070] In this embodiment, distributed rate limiting verification may include at least one of query per second (QPS) rate limiting verification and concurrency rate limiting verification.

[0071] QPS rate limiting focuses on controlling the rate of task requests per unit of time, making it suitable for real-time scenarios sensitive to response latency. It can prevent a sudden surge of requests from overloading the agent. Concurrency rate limiting focuses on controlling the number of task requests that are simultaneously executing. It is suitable for long-duration scenarios, preventing a large number of task requests from simultaneously consuming computing resources and exhausting them. The two rate limiting methods can be flexibly combined according to task type (i.e., AND / OR relationship) to form a dual guarantee of rate and concurrency, adapting to the differentiated needs of rate limiting granularity in different scenarios.

[0072] In some examples, QPS rate limiting verification can be implemented based on the token bucket algorithm. The token bucket generates a fixed number of tokens per second. Before consuming a task request, it must obtain a token from the token bucket. If successful, consumption is allowed; otherwise, it waits or is discarded. The token bucket achieves distributed consistency by executing Lua scripts in the distributed cache Redis. The Lua scripts atomically execute token generation, acquisition, and consumption logic, ensuring rate limiting accuracy in multi-node concurrent scenarios. Regarding token bucket capacity configuration, the bucket capacity can be set to twice the QPS limit, thus supporting short-term bursts of traffic and allowing temporary exceedances of the QPS limit when sufficient tokens are accumulated, but ensuring the long-term average rate does not exceed the set upper limit. When token acquisition fails, there are two handling strategies: one is to place the task request in a waiting queue and retry after the next token generation cycle; the other is to directly trigger discard logic based on the task's attributes (e.g., low priority or queuing time exceeding a threshold), releasing resources from the task request scheduling system.

[0073] In some examples, concurrency limiting verification can be implemented using a semaphore mechanism. A maximum concurrency can be set (e.g., maxConcurrency=100, meaning a maximum of 100 task requests can be in execution at any given time). A task request must acquire a semaphore before execution and release it upon completion. The semaphore can achieve distributed consistency by atomically incrementing and decrementing the concurrency counter using Redis's INCR and DECR commands. INCR increments the counter when a task request begins execution, and DECR decrements it when execution ends. To prevent semaphores from being held for extended periods, a timeout mechanism can be introduced: if a task request fails to acquire a semaphore within a preset timeout period (i.e., the current concurrency limit has been reached), the task request can enter a short-term waiting queue to continue trying, or its discard logic can be triggered directly based on task attributes. Regarding semaphore release timing, regardless of whether a task request succeeds or fails, the semaphore must be released promptly upon completion to avoid semaphore leakage, continuous accumulation of the concurrency counter due to abnormal conditions, and ultimately preventing subsequent task requests from acquiring the semaphore.

[0074] In these examples, QPS token bucket rate limiting can control the number of task requests entering the execution phase within a unit of time within the capacity of the agent, preventing downstream agent services from being overloaded due to sudden increases in request rates; concurrency semaphore rate limiting can control the number of task requests simultaneously consuming computing resources (such as GPU resources) within a preset limit, preventing service unavailability caused by resource exhaustion. With the combined effect of the two rate limiting methods, high-priority task requests can be given priority in both rate and concurrency dimensions, and their execution latency can be effectively reduced compared to infinite flow scenarios; when the traffic of the task request scheduling system surges, low-priority task requests will be prioritized and discarded or delayed due to token acquisition failure or semaphore waiting timeout, thereby reserving sufficient execution resources for high-priority task requests and ensuring the success rate of core task requests.

[0075] In some embodiments, the method further includes: detecting the backlog length of existing task requests in the first-in-first-out queue; when the backlog length exceeds a preset length threshold, distributing newly received task requests to a priority sorting queue; reading existing task requests in the first-in-first-out queue, calculating the priority of existing task requests according to a configuration mapping table, and distributing existing task requests to the priority sorting queue.

[0076] In this embodiment, the First-In-First-Out (FIFO) queue operates according to the FIFO rule. Task requests are queued and dequeued in the order they enter the queue. The FIFO queue itself does not have custom sorting capabilities; the execution order is determined solely by the time the task requests enter the queue. It does not dynamically rearrange the requests based on their importance or business priority. All task requests are treated the same. Task requests that enter the queue earlier are retrieved and executed first. Even if a task that enters the queue later has higher business value or more urgent time requirements, it must wait for all preceding task requests to be processed before it can be scheduled.

[0077] The system is configured with first-in-first-out queues and priority queues. The currently active task queue is manually selected and switched by the operations and maintenance personnel based on the characteristics of the business period.

[0078] During peak periods of high business request volume, operations and maintenance personnel can manually switch to enable the priority sorting queue. The system completes task attribute parsing for all normal business task requests, calculates priorities through a configuration mapping table and binds them to task requests, and distributes priority-bearing task requests to the priority sorting queue. Within the queue, tasks are sorted from high to low priority. Consumers read and execute tasks in priority order, enabling priority scheduling of high-value business tasks in peak scenarios and preventing high-priority requests from being blocked by a massive number of ordinary requests.

[0079] During periods of low business demand, operations and maintenance personnel can manually switch to the first-in-first-out (FIFO) queue. Task requests will no longer undergo priority calculation and marking. All task requests will be directly distributed to the FIFO queue and processed serially according to the order of enqueueing, reducing the system computing power overhead caused by priority sorting operations and simplifying the scheduling logic.

[0080] In real-world scenarios, during holidays (such as New Year's Day and Spring Festival) or sudden trending events, a large number of task requests flood into the task request scheduling system within a short period, and the number of backlogged task requests in the First-In-First-Out (FIFO) queue can quickly rise to tens of thousands. The FIFO queue, employing a FIFO mechanism, cannot differentiate between the user value of tasks. Tasks from high-value users (such as SVIP members) are queued equally with those from low-value users, leading to a similarly degraded user experience for both. Therefore, this disclosure provides a mechanism for dynamically switching to a priority queue mode. When the backlog exceeds a length threshold, a queue transition is automatically triggered, ensuring that task requests from high-value users are processed first, thus avoiding a situation where all users experience poorly.

[0081] It should be noted that the priority of the aforementioned existing task requests is calculated in the same way as the priority calculated in step S220 of Embodiment 1. Both are based on the same preset configuration mapping table and calculated according to the user attributes of the task requests (a combination of dimensions such as region, user type, membership level, and request time period). The only difference between the two is the application object: the priority calculated in step S220 is for newly received task requests and is calculated before the tasks are distributed to the priority sorting queue; the second priority is for existing task requests that have been backed up in the first-in-first-out queue and is calculated in batches after the queue conversion process is started, so that the existing task requests can be prioritized according to their task attributes and then distributed to the priority sorting queue along with their corresponding priorities.

[0082] The first-in, first-out (FIFO) queue produces and consumes tasks in the order of request time. When the number of backlogged task requests exceeds a length threshold, a queue transition process is triggered. Backlog length detection can be performed periodically by the execution layer monitoring module, which reads the queue length metric to determine the current backlog status. The length threshold can be dynamically configured based on the agent's actual processing capacity. For example, if the agent can process 100 task requests per second, the length threshold can be set to 10,000 (corresponding to approximately 100 seconds of backlog). If the agent's processing capacity changes (e.g., after scaling up to process 200 task requests per second), the length threshold can be dynamically adjusted through the configuration center without restarting the service.

[0083] In some examples, queue transformation can be divided into two steps. First, newly received task requests are routed to the priority queue, while a unified rate-limiting quota is applied to both the FIFO and priority queues. This unified rate-limiting quota means that the FIFO and priority queues share the same QPS token bucket and concurrency semaphore quotas, rather than maintaining separate rate-limiting quotas. This prevents the combined consumption of both from exceeding the task request scheduling system's capacity limit. For example, if the FIFO queue consumes 80% of the concurrency quota, the priority queue can only use the remaining 20%, ensuring that overall resource consumption never exceeds the preset limit. Second, existing task requests are read in batches from the FIFO queue. The task attributes of these requests are parsed, the priority of each request is calculated based on the configuration mapping table, and then the existing task requests are sent to the priority queue. Batch reading can be done by reading 100 requests at a time using Redis's LRANGE command, and then deleting the read requests using the LREM command after processing, ensuring that the same task request is not consumed repeatedly. Once all existing task requests in the FIFO queue have been imported into the priority queue, the FIFO queue can be marked as empty, and all subsequent task requests will be processed through the priority queue.

[0084] In other examples, the task import process can support resuming interrupted downloads to handle potential network fluctuations, service restarts, and other anomalies during the import process. The specific implementation of resuming interrupted downloads is as follows: the import progress is recorded in Redis, with the key name designed as `importProgress:{queueName}` (e.g., `importProgress:normalQueue`), and the value being the number of successfully imported task requests. This key is updated after each batch of imports is completed. If the import fails, it can resume from the breakpoint (i.e., the last recorded number of imported requests), avoiding the need to repeatedly import processed task requests from the beginning and preventing multiple executions of task requests due to duplicate imports. Regarding priority backfilling, if some task requests do not record user identifiers upon submission (e.g., legacy task requests or anonymous task requests), and user attributes cannot be obtained through user identifiers, priority cannot be calculated correctly. For such task requests, they can be assigned a default priority (e.g., priority 8, i.e., medium priority) and sent to a priority queue. This ensures that these task requests are not blocked in the import process due to the inability to calculate priorities, and also prevents high-value user task requests from being affected by being assigned excessively high priorities.

[0085] In these examples, the mechanism of converting the first-in-first-out queue to a priority queue can be used to deal with queue backlog scenarios, prioritize the processing of high-value user task requests with limited resources, and avoid poor user experience for all users.

[0086] Furthermore, embodiments of this disclosure also provide a queue transition recovery mechanism to form a complete switchover-recovery closed loop. When the backlog length decreases below the recovery length threshold as task requests are gradually consumed, newly received task requests can be rerouted to the first-in-first-out queue.

[0087] Through this switch-recovery closed-loop mechanism, the task request scheduling system can automatically switch to priority scheduling mode during peak backlog periods to ensure high-value user experience, and automatically revert to first-in-first-out queue mode after the backlog subsides, reducing the complexity of the task request scheduling system. No manual intervention is required, and adaptive switching of scheduling modes is achieved.

[0088] In some embodiments, the method further includes: detecting the queuing time of the task request in the priority queue; detecting whether the task request meets the discarding conditions based on the task attributes, queuing time, and priority; if the task request meets the discarding conditions, removing the task request from the priority queue and returning a discard response to the sender of the task request.

[0089] In this embodiment, the design goal of the discarding strategy is to proactively identify and release queue positions and rate-limiting quotas occupied by low-value task requests when the agent's computing resources are limited and task requests are severely backlogged, rather than passively waiting for task requests to time out naturally or for the task request scheduling system to crash due to resource exhaustion. By proactively discarding low-value task requests, the agent can concentrate its limited computing resources on high-value tasks, improving the success rate of core task requests without changing the overall processing capacity, and preventing a cascading effect where a large backlog of low-value task requests causes high-value task requests to be equally affected.

[0090] There are two ways to trigger discard detection. The first is periodic proactive detection: The priority queue is scanned at set intervals, and each task request is checked to see if it meets the discard criteria. Requests that meet the criteria are removed from the queue. This method is suitable for scenarios where queuing time exceeds limits, allowing for timely clearing of long-accumulated low-value task requests. The second is passive detection during task request consumption: After reading a task request from the priority queue but before invoking the agent, the task request is checked to see if it meets the discard criteria. If it does, it is marked as discarded without invoking the agent. This method is suitable for real-time discarding scenarios when the agent is overloaded, providing a final filter just before task requests are executed. These two detection methods can be combined to form a dual discard guarantee at both the queue and consumption layers.

[0091] Drop conditions can be configured based on multiple dimensions, supporting combinations of conditions such as queuing time, priority level, tags, task type, region, user type, membership level, and time period. Drop conditions can also support logical expression combinations; for example, queuing time exceeding a threshold and priority below a threshold, or a non-core task and a new user, can trigger drop. Regarding the priority order of drop conditions, queuing time can be prioritized over user level. That is, when a task request's queuing time exceeds the threshold, the drop condition is met regardless of the user's level. Only when the queuing time does not exceed the threshold is the drop condition further determined based on user level and other conditions. This priority order ensures that timed-out task requests are cleaned up first, preventing them from occupying queue positions and affecting the processing of subsequent task requests. Furthermore, drop conditions also support dynamic hot updates via the configuration center. Similar to the configuration mapping table hot update mechanism, changes to drop condition configurations by operations personnel take effect within seconds without restarting the service, allowing for adjustments to the drop strategy based on actual task requests.

[0092] Dropping operations can be performed separately on the sending and consuming ends. Sending-end dropping means that upon receiving a task request, it immediately checks whether the dropping conditions are met. If the conditions are met, a drop response is returned directly, without sending the task request to the queue, reducing queue backlog and storage pressure. Sending-end dropping can also support degradation strategies: when the agent is overloaded, strict dropping conditions are automatically enabled, reducing the priority threshold and thus more aggressively dropping low-value task requests; when the agent's load returns to normal, the lenient dropping rules are automatically restored, reducing the dropping of low-priority task requests. Consuming-end dropping means that after the consumer retrieves a task request from the queue, it checks whether the task request meets the dropping conditions before calling the agent. If the conditions are met, the task request is directly marked as dropped, releasing the queue position and rate-limiting quota. Consuming-end dropping can simultaneously save dropping records, including the task request ID, dropping reason (e.g., queue timeout, task request scheduling system overload, low priority, etc.), and dropping time. These records are written to key-value storage for querying by the task request scheduling system. The task request scheduling system can periodically query discard records and decide whether to resubmit the task request based on the reason for the discard. For example, a task request that was discarded due to agent overload can be resubmitted after the agent's load returns to normal.

[0093] In some examples, typical drop-out scenarios include the following: Scenario 1: In real-time dialogue scenarios, task requests queued for more than 10 seconds are dropped to avoid long user wait times. In real-time dialogue scenarios, users expect millisecond-level response times. If a task request queues for more than 10 seconds in the priority queue, the user experience has severely degraded. Continuing to process the task request is pointless; directly returning a drop response allows users to retry or use other methods to obtain service. Scenario 2: In video generation scenarios, task requests queued for more than 3 days are dropped to release backlogged resources. Video generation scenarios allow for longer processing times, but if a task request queues for more than 3 days, users may have given up or resubmitted the task request. Continuing to process the task request not only wastes resources but also occupies queue space, affecting the processing of subsequent task requests. Scenario 3: During major promotional events, non-core task requests are dropped to ensure the stability of core task requests. During major promotional events, traffic for core task requests (such as product recommendations and order processing) surges. Non-core task requests (such as user profile updates and data analysis report generation) can be temporarily dropped, concentrating computing resources on core task requests. Scenario 4: When the agent is overloaded, task requests from regular members and new users are dropped to prioritize high-value user experiences. Scenario 5: Cleanup after a major promotional event. After the event ends, a large number of event-related task requests (such as video generation tasks submitted during the event) may still be backed up in the priority queue. The timeliness of these task requests has been significantly reduced. Batch drop rules can be configured to drop task requests with tags containing the event identifier and submission times earlier than the event end time, quickly releasing backlogged resources and restoring the task request scheduling system to normal operation.

[0094] In these examples, when the agent is overloaded, it can automatically clear low-priority or timed-out task requests to prevent avalanche effects and ensure the success rate of processing core task requests.

[0095] It's important to note that the intelligent discarding strategy and the distributed rate limiting strategy work synergistically to ensure the stability of the task request scheduling system. They work together from different dimensions to maintain stability: the distributed rate limiting strategy controls the number of task requests entering the execution phase per unit time (rate-level control), preventing sudden surges in traffic from overwhelming downstream intelligent agent services; the intelligent discarding strategy controls the total number of task requests waiting to be executed in the priority queue (stock-level control), preventing low-value task requests from occupying queue positions for extended periods, thus preventing high-value task requests from being unable to be processed in a timely manner. Through their combined effect, the task request scheduling system can both control the processing rate to not exceed the downstream carrying capacity and ensure that the queue is always dominated by high-value task requests. Therefore, when the overall processing capacity of the task request scheduling system is limited, it maximizes the allocation of limited resources to core task requests and high-value users.

[0096] In some embodiments, distributing a task request carrying a corresponding priority to a priority sorting queue includes: determining a target storage medium that matches the task type; and distributing the task request carrying the corresponding priority to the priority sorting queue corresponding to the target storage medium.

[0097] In this embodiment, the motivation for designing a differentiated storage strategy lies in the different performance requirements of storage media for real-time and long-duration scenarios. Real-time scenarios are extremely sensitive to response latency, requiring message storage and key-value storage to have the lowest possible read / write latency. If distributed storage media requiring network I / O are used, the additional network latency may cause the overall response time to exceed the user's acceptable range. Long-duration scenarios, on the other hand, have higher storage capacity requirements, needing to support the accumulation of millions or even hundreds of millions of task requests. If memory-based storage media are used, the capacity limit will severely restrict the task request handling capacity of the task request scheduling system. Therefore, using the same storage medium uniformly cannot simultaneously meet the needs of both types of scenarios. It is necessary to dynamically match the target storage medium according to the task attributes to fully leverage the performance advantages of different storage media.

[0098] In this embodiment, the selection logic for the target storage medium can be predefined in a configuration file, defining the mapping relationship between task attributes and storage media. For example, when the task type is a real-time task, the corresponding storage medium can be an asynchronous message passing library, and the key-value storage can be an in-memory key-value cache; when the task type is a long-running task, the corresponding storage medium can be a distributed message queue, and the key-value storage can be a combination of an in-memory multi-model database and table storage; when the task type is a batch task, the corresponding storage medium can be a distributed message queue, and the key-value storage can be table storage. This configuration file can also be hot-updated through a configuration center, allowing operators to adjust the mapping relationship between task types and storage media according to actual needs, switching storage strategies without modifying the code.

[0099] In some examples, for long-running task requests, the target storage medium can be either an in-memory multi-model database or a distributed message queue, supporting the accumulation of millions or even hundreds of millions of task requests. Key-value storage can be written to both an in-memory multi-model database and a table store by default, employing a strategy of writing to the in-memory multi-model database first and then asynchronously writing to the table store.

[0100] In other examples, for real-time task requests, message storage can choose an asynchronous messaging library, reducing latency through in-memory queues. Asynchronous messaging libraries use in-memory messaging mechanisms where message production and consumption are completed in memory, eliminating the need for network I / O. End-to-end latency can reach the microsecond level, significantly reducing latency compared to in-memory multi-model databases or distributed message queues that require network I / O. Key-value storage can write to an in-memory key-value cache, further reducing read / write latency and improving response speed in real-time scenarios.

[0101] In these examples, a memory queue can be selected as the priority queue for real-time types to reduce link latency, while a distributed message queue that supports massive stacking can be selected as the priority queue for long-duration types, giving full play to the performance advantages of different storage media and realizing the configurability and replaceability of the storage layer.

[0102] In some embodiments, the method further includes: receiving an update instruction for the configuration mapping table through a configuration center, and performing a hot update on the configuration mapping table.

[0103] In this embodiment, the necessity of the hot update mechanism lies in the extremely frequent changes in task requirements of LLM application scenarios. During major promotional events and sudden trending events (such as major sporting events or social hot topics), the composition of high-value user groups and priority strategies may need to be adjusted within minutes. If each adjustment of the priority strategy requires a complete development process including code modification, code review, testing, deployment, and service restart, it typically takes 1 to 2 days, far from meeting the rapid response requirements of task requests. Therefore, implementing hot updates of priority strategies through a configuration center, allowing operations personnel to bypass the development process and directly modify configurations with effects taking effect in seconds, is a key technical means to solve the above problems.

[0104] There are two ways to obtain update instructions. The first is for operations personnel to manually modify the configuration through the configuration center's web management interface. For example, before a major promotional event, the priority of VIP users might be temporarily adjusted from 4 to 2, and then restored to the original configuration after the event. The second is to automatically trigger configuration changes through scheduled tasks. For example, a scheduled rule can be set in the configuration center: automatically switch to peak-hour configuration at 9:00 AM daily (prioritizing core task requests and high-value users), and automatically switch to off-peak-hour configuration at 6:00 PM daily (restoring the default priority allocation). This allows the priority strategy to automatically adjust over time, reducing manual intervention. These two methods can be used in combination: scheduled task requests handle routine time-based adjustments, while manual modifications address unexpected task demands, jointly ensuring consistency between the priority strategy and actual task requirements.

[0105] After operators modify the configuration mapping table through the configuration center's management interface, the configuration center pushes configuration change notifications to each node of the task request scheduling system. Nodes pull the latest configuration and apply it in real time, without requiring service restarts; configuration effectiveness can be achieved within seconds. Regarding high availability, the configuration center itself can be deployed in a cluster, using master-slave replication and automatic failover mechanisms to avoid single points of failure and ensure continuous availability of the configuration push service. For local caching fault tolerance, each node can persist the configuration to its local disk or memory cache after its initial configuration pull. When the configuration center experiences a brief downtime (e.g., network jitter or configuration center node restart), each node can continue using the last valid configuration cached locally, ensuring the normal execution of task request scheduling is not affected by the configuration center's unavailability. Once the configuration center becomes available again, the nodes automatically re-establish connections and synchronize the latest configuration, achieving transparent recovery from configuration service failures.

[0106] In some examples, the configuration center can use a long polling mechanism to push configuration information. A node sends a long polling request to the configuration center. Upon receiving the request, if there are no current configuration changes, the configuration center suspends the request (keeping the connection open without immediately returning) until a configuration change occurs, at which point it immediately responds and pushes the latest configuration. If no configuration change occurs within a preset timeout period, an empty response is returned. Upon receiving the empty response, the node immediately re-initiates the next long polling request, and this process repeats. After receiving a configuration change notification, the node updates its local cache and immediately applies the new configuration.

[0107] In other examples, the configuration center can also employ a publish-subscribe mechanism. Nodes subscribe to configuration change topics, the configuration center publishes configuration change messages to the topics, and all subscribing nodes pull the latest configuration upon receiving the message. The publish-subscribe mechanism is particularly suitable for scenarios with multiple data center deployments: when a task request scheduling system is deployed in multiple geographically dispersed data centers (e.g., one data center each in North China, East China, and South China), after the configuration center publishes a configuration change message, the distributed message queue can route the message to the subscribing nodes in all data centers, ensuring that nodes in different data centers can receive configuration change notifications synchronously, achieving consistent updates of the global configuration, and avoiding differences in scheduling strategies due to inconsistent configurations across different data centers.

[0108] In some examples, the publish-subscribe mechanism can also be used in conjunction with the long polling mechanism, with the publish-subscribe mechanism as the primary push channel and the long polling mechanism as the backup push channel. When the message middleware fails, it automatically degrades to the long polling mechanism, further improving the reliability of configuration push.

[0109] In some examples, operators can adjust the priority of task requests at any time according to task requirements. For example, during a major promotion, the priority of VIP users can be temporarily increased, and the original priority can be restored after the event ends.

[0110] In these examples, the response time for priority adjustment has been reduced from days to seconds, enabling rapid response to sudden task demands such as major promotional events and trending topics.

[0111] It should be noted that the hot update mechanism is not limited to the configuration mapping table; it can also cover all configurable items of the task request scheduling system at runtime, including but not limited to: rate limiting parameters (QPS limit, maximum concurrency, token bucket capacity and semaphore quota for each priority queue), drop conditions, length thresholds, and storage media mapping relationships (mapping configuration between task types and storage media). All of the above configuration items can be hot-updated through the configuration center, taking effect within seconds without restarting the service, giving the task request scheduling system comprehensive runtime configurability capabilities and enabling it to flexibly respond to various sudden task scenarios.

[0112] In some embodiments, the method further includes: obtaining an event routing relationship pre-configured for the task type; and updating the task status of the task request based on the event routing relationship.

[0113] This embodiment further describes the specific implementation of configurable orchestration of executors under Event-Driven Architecture (EDA). The motivation behind the design of Event-Driven Architecture (EDA) is that the execution flow of task requests for different task types varies significantly. For example, generating a college entrance examination DRS in-depth research report requires first decomposing the report task request into multiple sub-task requests, and then executing each sub-task request one by one; while video generation tasks do not require a planning phase and can directly call the video generation agent to execute the task request. If a fixed execution flow is used, it cannot flexibly adapt to the differentiated needs of different task scenarios, and each new task scenario requires modification of the execution layer code. Therefore, this embodiment drives the orchestration logic of the executors through pre-configured event routing relationships. Different task scenarios only need to modify the event routing configuration to complete the integration, without modifying the core code.

[0114] Event routing relationships can be retrieved when a task request enters the execution queue. This avoids repeatedly querying the configuration file every time an event is triggered, reducing the performance overhead of configuration queries. The event routing relationship configuration file can be stored in the configuration center in JSON format, supporting hot updates. After operations personnel modify the configuration, newly received task requests will use the new event routing configuration, while task requests already in execution will continue to use the original configuration, preventing interruptions to the execution process of task requests due to configuration changes.

[0115] In some examples, updating the task status of a task request may include: updating the task status of the task request and publishing an execution completion event upon detecting a success event published by the agent when the task request was successfully executed; or republishing the task request to a priority queue upon detecting a failure event published by the agent when the task request was failed.

[0116] Event-driven architecture can include four types of executors: planners, execution workers, exception handling workers, and completion workers. Each execution unit is logically independent, collaborates through an event-driven mechanism, and can scale independently. Executors communicate with each other via a distributed message queue, rather than through direct calls, achieving decoupling between services: the executor publishing an event does not need to know the specific addresses and implementation details of the executors subscribing to the event; it only needs to publish the event message to the corresponding topic in the message queue, which then routes the message to all executor instances subscribing to that topic. This decoupling design allows each executor to scale independently: for example, when the concurrency of video generation task requests increases, the number of execution worker instances can be scaled up only without simultaneously scaling up the planner, exception handling, and completion workers, thus achieving on-demand scaling and reducing resource costs. In terms of deployment, each executor can be containerized and combined with an automatic scaling strategy, automatically adjusting the number of instances based on the backlog of messages in the corresponding message queue topic to achieve elastic scaling.

[0117] The task planner can be used for task request decomposition, dependency analysis, and resource estimation, and publishes a successful execution event upon completion. A typical application scenario for the task planner is complex scenarios that require task decomposition before execution. For example, generating a DRS in-depth research report requires first decomposing the report task request into multiple sub-query task requests (such as background research, data analysis, and conclusion writing), analyzing the dependencies between these sub-task requests (e.g., conclusion writing depends on data analysis completion), and estimating the computational resources required for each sub-task (e.g., estimating GPU memory requirements) so that the execution layer can allocate resources appropriately. For simple task requests that do not require a planning phase (e.g., video generation task requests), the task planner can be skipped through event routing configuration, and the execution worker can directly subscribe to the task queue and execute the task. The execution worker can subscribe to a successful plan event and call the agent to execute the task request. Upon successful execution, it publishes a successful execution event; upon failure, it publishes an execution failure event. The way the executor calls the agent can be flexibly configured according to the task attributes: For real-time task requests, a synchronous HTTP call can be used to directly wait for the agent to return the result; for long-running task requests, an asynchronous message call can be used to send a task message to the agent and return immediately. After the agent completes the execution, it notifies the executor through a callback or message queue to avoid occupying thread resources due to long waiting time.

[0118] The exception handling worker can subscribe to execution failure events and determine whether to republish the task request to the execution queue based on the retry policy. For example, if the number of retries is less than a set number, the request is republished; otherwise, the task request is marked as a final failure. Regarding the retry policy, the exception handling worker can employ an exponential backoff strategy to avoid retry storms. For example, the first retry interval is 1 second, the second is 2 seconds, the third is 4 seconds, and so on, with each retry interval doubling the previous one. The exponential backoff strategy provides sufficient recovery time for the agent service when it experiences a brief failure, preventing a massive influx of retry requests that could cause the agent service to collapse. The retry interval can be configured through the configuration center, supporting different retry intervals and maximum retries for different task types. The completion worker can subscribe to execution success events, persistently store the execution result, update the task status to "completed," and publish an execution completion event. The task system subscribes to this event to obtain the task request result. The content that can be persistently stored by the worker can include: execution results (the complete output returned by the agent), execution time (the total time from the submission of the task request to the completion of the execution, as well as the time details of each stage), task status (completed or ultimately failed), completion time (the timestamp of the completion of the task request), etc. All of this information is written to the KV storage for the task system to query and use for subsequent data analysis.

[0119] In some examples, the execution flow of different task scenarios can be defined through a JSON configuration file, which specifies the event routing relationship of each executor.

[0120] For example, the DRS task request for the National College Entrance Examination (NCEE) can be configured with an execution flow of Plan → Execute → Complete, such as... Figure 3 As shown, the HTTP request parser receives external task requests and pushes them to the planning queue. The planning worker consumes the task requests in the planning queue, completes the task planning process, and then sends them to the execution queue for the execution worker to execute. Task requests that encounter exceptions during execution flow into the exception task queue, where the exception task worker performs exception repair. Task requests that fail to resolve errors during the planning phase are also directly returned to the exception task queue. Task requests that fail to resolve exceptions are stored in the failure task queue, and task requests that complete execution normally enter the completion task queue. Data from the failure task queue and the completion task queue are sent to the completion worker to complete the lifecycle and archive the status of the task requests.

[0121] Video generation task requests can be configured to execute → complete, skipping the planning phase, such as... Figure 4 As shown, the HTTP request parser directly sends the task request to the execution queue, where the execution worker directly performs the task calculation. Task requests that are executed normally enter the completed task queue, while task requests that are executed abnormally enter the abnormal task queue, where the abnormal task worker performs exception handling. Task requests that fail to handle exceptions enter the failed task queue, while task requests that are completed normally enter the completed task queue. The data from the failed task queue and the completed task queue are sent to the completed worker to complete the lifecycle of the completed task request and archive its status.

[0122] Adding new task types requires no code modification; simply configuring the executor's event routing relationships completes the integration. The integration cycle for new task types has been shortened from weekly to daily. Regarding hot configuration updates, after modifying the event routing configuration through the configuration center, newly received task requests will use the new event routing configuration. Meanwhile, task requests already in execution will continue to execute using the original configuration because the event routing relationships are cached in the task context. This avoids interruptions to the execution of task requests due to configuration changes, ensuring consistency in task request execution.

[0123] It is understood that the storage structure of the above configuration mapping table is not limited to hash tables; other storage formats such as relational databases and key-value databases can also be used. The underlying implementation of the above priority-sorting queue is not limited to Redis sorted sets; other data structures or message brokers that support priority sorting can also be used. The distributed implementation of the above token bucket algorithm is not limited to Redis Lua scripts; other distributed caching implementations that support atomic operations can also be used. The above configuration center is not limited to Diamond; other configuration management components that support hot configuration updates can also be used. The deployment method of the above executors is not limited to single-machine multi-threaded deployment; distributed multi-node deployment can also be used. Multiple instances of each executor work collaboratively through the same message queue topic. The message queue is responsible for distributing event messages to each instance. Each instance consumes messages in parallel and executes the corresponding processing logic. Horizontal scaling by increasing the number of instances is supported to cope with the large-scale task request processing needs in high-concurrency scenarios. During horizontal scaling, instances do not need to communicate directly; automatic load balancing of task requests is achieved through the message queue. The scaling process is transparent to task requests and requires no configuration modification or service restart.

[0124] In these examples, additional configuration and reading of event routing relationships specific to various task types are performed. Based on preset routing rules, the entire lifecycle state update of task requests is driven. On one hand, differentiated state transition branches can be configured for different task types such as real-time interaction and long-duration generation, enabling targeted state updates for different events such as task request execution, rate limiting interception, retry failure, and execution completion. This adapts to differentiated process management of intelligent agent task requests across multiple scenarios. On the other hand, the scheduling logic is decoupled from the event routing configuration. When adding a new task type, only the corresponding routing relationship needs to be configured to complete state management adaptation without modifying the underlying scheduling code, improving the iteration efficiency of task types. Simultaneously, the entire process task state is recorded completely and in a standardized manner, providing complete state data support for task request retries, queuing time statistics, fault tracing, and rate limiting monitoring. This further improves the task request scheduling system's unified management capability across the entire intelligent agent task request chain, balancing scheduling scalability, operational observability, and adaptability to multiple task types.

[0125] <Second Embodiment> This embodiment provides an electronic device. Figure 5 A schematic diagram of the hardware structure of the electronic device is shown.

[0126] like Figure 5As shown, the electronic device 500 includes a processor 510 and a memory 520. The memory can be used to store computer programs, and the processor can be used to retrieve the computer programs from the memory to execute any method embodiment of this disclosure. The processor can be one or more, and these processors can execute instructions individually or jointly. Similarly, the memory can be one or more, and these memories can store the aforementioned computer programs individually or jointly.

[0127] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods in any embodiment of this disclosure. Optionally, the computer-readable storage medium may be a non-transitory storage medium, but is not limited thereto; it may also be a temporary storage medium.

[0128] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0129] This disclosure may be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having a computer-readable program loaded thereon for causing a processor to implement any of the methods in the foregoing embodiments of this disclosure.

[0130] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media may include, for example, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), compact disc-read-only memory (CD-ROM), digital versatile disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any combination thereof. The computer-readable storage medium used herein is not to be interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0131] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include one or more of copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to computer-readable storage media in the respective computing / processing device.

[0132] The computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object programs written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and conventional procedural programming languages ​​(such as the "C" language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network (e.g., a local area network or a wide area network), or it may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays, or programmable logic arrays, can execute computer-readable program instructions to implement various aspects of the embodiments of this disclosure by utilizing state information from the computer-readable program instructions.

[0133] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0134] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0135] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0136] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It should be noted that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are all equivalent.

[0137] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of this disclosure is defined by the appended claims.

Claims

1. A method for task request scheduling of an agent, characterized in that, The method includes: Receive a task request to be executed and parse the task attributes of the task request; Based on a preset configuration mapping table, the priority of the task request is determined according to the task attributes; wherein, the configuration mapping table stores the mapping relationship between task attributes and priorities; Distribute the task requests carrying the corresponding priorities to the priority queue; The task requests in the priority queue are read in descending order of priority, and the agent is invoked to execute the task requests read from the priority queue.

2. The method of claim 1, wherein, The task attributes include any one or more of the following: region, user type, membership level, request time period, and task type.

3. The method according to claim 1, characterized in that, The step of reading task requests from the priority queue in descending order of priority and invoking the agent to execute the task requests read from the priority queue includes: When reading the task requests in the priority queue in descending order of priority, distributed rate limiting verification is performed on the task requests read from the priority queue. The intelligent agent is invoked to execute the task request that has passed verification.

4. The method according to claim 3, characterized in that, The task attributes include the task type; the distributed rate limiting verification performed on the task requests read from the priority queue includes: Based on the task type, a query rate limiting check is performed on the task requests read from the priority queue using the token bucket algorithm; and / or, a concurrency rate limiting check is performed on the task requests read from the priority queue using the semaphore mechanism.

5. The method according to claim 1, characterized in that, The method further includes: Detect the backlog length of existing task requests in the first-in-first-out queue; When the backlog length exceeds a preset length threshold, newly received task requests are distributed to the priority sorting queue; Read the existing task requests in the first-in-first-out queue, calculate the priority of the existing task requests according to the configuration mapping table, and distribute the existing task requests to the priority sorting queue.

6. The method according to claim 1, characterized in that, The method further includes: Detect the queuing time of the task request in the priority queue; Based on the task attributes, the queuing time, and the priority, it is determined whether the task request meets the discarding condition; If the task request meets the discarding condition, the task request is removed from the priority queue, and a discard response is returned to the sender of the task request.

7. The method according to claim 1, characterized in that, The task attribute also includes a task type, and the step of distributing the task request carrying the corresponding priority to a priority queue includes: Determine the target storage medium that matches the task type; The task request carrying the corresponding priority is distributed to the priority queue corresponding to the target storage medium.

8. The method according to claim 1, characterized in that, The method further includes: The configuration center receives update instructions for the configuration mapping table and performs hot updates on the configuration mapping table.

9. The method according to claim 1, characterized in that, The task attribute also includes the task type, and the method further includes: Obtain the event routing relationships pre-configured for the task type; The task status of the task request is updated based on the event routing relationship.

10. An electronic device, characterized in that, It includes a memory and a processor, the memory being used to store a computer program; the processor being used to execute the computer program to implement the method according to any one of claims 1-9.

11. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the method according to any one of claims 1-9.

Images