Coroutine pool scheduling method and electronic device

Abstract

The application discloses a coroutine pool scheduling method and electronic equipment, relates to the technical field of data processing, and realizes non-competition distribution of tasks through a lock-free task queue, avoids global lock competition, makes the coroutine obtain and return in the processor core through a local cache pool, reduces the overhead of cross-processor migration and frequent creation and destruction, binds a special coroutine pool through task information, isolates different business resources from each other, avoids resource contention, and guarantees local scheduling efficiency while taking into account global resource utilization through the acquisition sequence of the local cache pool, the special coroutine pool and the global coroutine pool, adjusts the coroutine distribution or dynamically expands and shrinks the capacity according to the number of tasks and the number of local idle coroutines, makes the coroutine resources change adaptively with the load, and avoids task backlog or resource idling. Therefore, the application can solve the problems of low scheduling efficiency, uneven resource allocation and poor load adaptability under high concurrency.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a coroutine pool scheduling method and electronic device. Background Technology

[0002] In high-concurrency data processing and network service scenarios, the number of tasks is large and their arrival is volatile. Traditional thread models incur significant overhead in creation, destruction, and context switching, resulting in high resource consumption. While coroutines offer advantages such as low user-mode switching overhead and low resource consumption, without unified management and scheduling, issues such as frequent creation and destruction, task distribution contention, uneven resource allocation, and insufficient adaptability to load changes can lead to decreased scheduling efficiency and uneven resource utilization. Therefore, achieving efficient scheduling and resource reuse of coroutines in high-concurrency scenarios is a technical problem that needs to be solved. Summary of the Invention

[0003] This application provides a coroutine pool scheduling method and electronic device to at least solve the problem of how to achieve efficient scheduling and resource reuse of coroutines in high-concurrency scenarios in related technologies.

[0004] This application provides a coroutine pool scheduling method, including: Construct a global coroutine pool, local cache pools for multiple processors, and a lock-free task queue; In response to the processing request of the target task, a dedicated coroutine pool associated with the target task is determined based on the task information of the target task. The local cache pool, the dedicated coroutine pool, and the global coroutine pool include coroutines used to process tasks. The coroutines in the local cache pool are allocated from the global coroutine pool. The target task is distributed to the corresponding lock-free task queue according to the preset task priority, and idle coroutines are obtained in the order of local cache pool, dedicated coroutine pool, and global coroutine pool, and the target task is executed based on the idle coroutines. Based on the number of tasks in the lock-free task queue and the number of idle coroutines in the local cache pool, adjust the distribution of idle coroutines between the local cache pool and the global coroutine pool, and / or dynamically expand or shrink the global coroutine pool. The coroutine that has completed the target task will be returned to the local cache pool for reuse.

[0005] This application also provides a coroutine pool scheduling device, including: The building unit is used to build a global coroutine pool, local cache pools corresponding to multiple processors, and lock-free task queues. The determining unit is used to respond to the processing request of the target task and determine the dedicated coroutine pool associated with the target task based on the task information of the target task. The local cache pool, the dedicated coroutine pool, and the global coroutine pool include coroutines for processing tasks. The coroutines in the local cache pool are allocated from the global coroutine pool. The distribution unit is used to distribute target tasks to the corresponding lock-free task queues according to preset task priorities. The acquisition unit is used to acquire idle coroutines in the order of local cache pool, dedicated coroutine pool, and global coroutine pool, and to execute the target task based on the idle coroutines; The adjustment unit is used to adjust the distribution of idle coroutines between the local cache pool and the global coroutine pool based on the number of tasks in the lock-free task queue and the number of idle coroutines in the local cache pool, and / or to dynamically expand or shrink the global coroutine pool. The reuse unit is used to return a coroutine that has completed its target task to the local cache pool for reuse.

[0006] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing the steps of any of the above-described coroutine pool scheduling methods when executing the computer program.

[0007] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described coroutine pool scheduling methods.

[0008] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described coroutine pool scheduling methods.

[0009] The coroutine pool scheduling method and electronic device of this application achieve non-contention-based task distribution through a lock-free task queue, avoiding global lock contention; it enables coroutines to be acquired and returned locally within the processor core through a local cache pool, reducing the overhead of cross-processor migration and frequent creation and destruction; it isolates different business resources by binding task information to a dedicated coroutine pool, avoiding resource contention; it ensures both local scheduling efficiency and global resource utilization by controlling the acquisition order of the local cache pool, dedicated coroutine pool, and global coroutine pool; and it adjusts the distribution of coroutines or dynamically scales up and down based on the number of tasks and the number of local idle coroutines, allowing coroutine resources to adapt to the load and avoiding task backlog or resource idleness. Therefore, this application can solve the problems of low scheduling efficiency, uneven resource allocation, and poor load adaptability under high concurrency. Attached Figure Description

[0010] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A flowchart illustrating a coroutine pool scheduling method provided in an embodiment of this application; Figure 2 A schematic diagram of the overall framework of a coroutine pool scheduling system provided in this application embodiment; Figure 3 A schematic diagram of task submission and binding pool allocation provided in an embodiment of this application; Figure 4 A schematic diagram of a dynamic scaling process provided in an embodiment of this application; Figure 5 This application provides a schematic diagram of an adaptive cache pool water level write-back process. Figure 6 This application provides a coroutine lifecycle state diagram. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0013] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0014] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0015] Figure 1 This document provides a flowchart illustrating a coroutine pool scheduling method as an embodiment of the present application. The coroutine pool scheduling method will be described in detail below, taking into account its flowchart.

[0016] Step 101: Construct a global coroutine pool, local cache pools for multiple processors, and a lock-free task queue.

[0017] In the embodiments of this application, the global coroutine pool is a resource set used to store idle coroutines that have not yet been allocated to the local cache pools of each processor. It serves as a resource buffer and fallback source for the entire scheduling system. When the local cache pool or the dedicated coroutine pool cannot provide idle coroutines, the scheduler can obtain coroutine resources from the global coroutine pool.

[0018] The local cache pool is a localized coroutine cache structure set up for each processor. The processor can be a physical core, a logical core, or a processing unit capable of independently executing scheduling logic within a central processing unit. Each processor has an independent local cache pool to store coroutines that have just completed their tasks and are in an idle state. This allows coroutines to return to the local cache pool of the current processor first after completing their tasks, achieving local reuse and reducing the migration of coroutines between different processors.

[0019] A lock-free task queue is a data structure used to store tasks to be executed. It supports multiple producers concurrently enqueuing tasks into the queue and multiple consumers concurrently dequeuing tasks from the queue. The entire enqueueing and dequeueing process can be completed without relying on a global lock, thereby eliminating the lock contention bottleneck in high-concurrency scenarios.

[0020] It should be noted that the coroutines in the local cache pool are allocated from the global coroutine pool. In other words, during system initialization or operation, idle coroutines in the global coroutine pool will be allocated to various local cache pools, which will then prioritize their management and reuse.

[0021] Step 102: In response to the processing request of the target task, determine the dedicated coroutine pool associated with the target task based on the task information of the target task. The local cache pool, the dedicated coroutine pool, and the global coroutine pool include coroutines used to process tasks. The coroutines in the local cache pool are allocated from the global coroutine pool.

[0022] In the embodiments of this application, the target task refers to the business task or computing task that needs to be scheduled for execution, which can come from various scenarios such as network request processing, real-time data processing, and asynchronous callback execution.

[0023] Task information is a set of data used to describe the attributes and resource requirements of a target task, including at least binding information used to indicate a dedicated coroutine pool.

[0024] A dedicated coroutine pool refers to a coroutine resource pool independently configured for a specific business, a specific tenant, or a task with a specific priority. It, along with the local cache pool and the global coroutine pool, constitutes a hierarchical system of coroutine resources. The local cache pool, dedicated coroutine pool, and global coroutine pool all contain coroutines used to process tasks. Identifying a dedicated coroutine pool through task information means that a task is explicitly associated with a specific dedicated coroutine pool at the submission stage. Therefore, during subsequent scheduling and execution, the scheduler can preferentially obtain coroutine resources from that dedicated coroutine pool. This ensures that coroutine resources, task queues, and scheduling logic are independent between different businesses, achieving resource isolation and fault domain isolation, which is beneficial for ensuring the service quality of core businesses.

[0025] Step 103: Distribute the target task to the corresponding lock-free task queue according to the preset task priority, and obtain idle coroutines in the order of local cache pool, dedicated coroutine pool, and global coroutine pool, and execute the target task based on the idle coroutines.

[0026] In the embodiments of this application, the preset task priority refers to the execution priority assigned to a task when it is submitted, such as high priority, medium priority, low priority, etc. Different priorities correspond to different lock-free task queues. High-priority tasks are assigned to the high-priority queue, medium-priority tasks are assigned to the medium-priority queue, and low-priority tasks are assigned to the low-priority queue.

[0027] When the scheduler acquires tasks, it does so in descending order of priority, ensuring that high-priority tasks can obtain coroutine resources and be executed first. After distributing the target task to the corresponding lock-free task queue, the scheduler acquires idle coroutines in the following order: local cache pool, dedicated coroutine pool, and global coroutine pool.

[0028] An idle coroutine is a coroutine that is idle and waiting to acquire a task and execute it. The acquisition order of an idle coroutine includes: first, attempting to acquire an idle coroutine from the local cache pool corresponding to the current scheduler, because coroutines in the local cache pool are located in the local cache of the current processor core, and the acquisition and reuse overhead is minimal; if there is no idle coroutine in the local cache pool, then further acquiring an idle coroutine from the dedicated coroutine pool associated with the target task based on the task information of the target task; if there is no idle coroutine in the dedicated coroutine pool either, then finally acquiring an idle coroutine from the global coroutine pool.

[0029] By employing a three-tiered resource acquisition order, coroutine reuse can be maximized, reducing cache coherency overhead caused by cross-processor migration. Furthermore, even when local resources are insufficient, task execution can still be guaranteed through dedicated and global pools. After acquiring an idle coroutine, the target task is executed based on that idle coroutine.

[0030] Step 104: Based on the number of tasks in the lock-free task queue and the number of idle coroutines in the local cache pool, adjust the distribution of idle coroutines between the local cache pool and the global coroutine pool, and / or perform dynamic scaling up and down processing on the global coroutine pool.

[0031] In the embodiments of this application, the number of tasks in the lock-free task queue reflects the current task load of the system. The more tasks there are, the more serious the backlog of tasks to be processed, and the more coroutine resources the system needs to accelerate task processing. The fewer tasks there are, the lighter the system load, and coroutine resources can be appropriately reduced to save memory and processor overhead.

[0032] The number of idle coroutines in the local cache pool reflects whether the processor has sufficient coroutine resources that can be immediately reused locally. When there are too many idle coroutines in the local cache pool, it indicates that there is redundancy in local resources. The excess idle coroutines can be written back to the global coroutine pool for use by other processors or dedicated coroutine pools. When there are not enough idle coroutines in the local cache pool, idle coroutines can be added to the local cache pool from the global coroutine pool to ensure local scheduling capabilities.

[0033] Dynamic scaling refers to automatically increasing or decreasing the number of coroutines in the global coroutine pool based on the number of tasks and the number of idle coroutines. When the task volume surges and the existing coroutines are insufficient to process tasks quickly, scaling up creates new coroutines and adds them to the global coroutine pool or local cache pool to improve system processing capacity. When the task volume drops sharply and a large number of coroutines are idle for a long time, scaling down destroys some idle coroutines and reclaims resources to the memory pool to save system resources.

[0034] It should be noted that adjusting the distribution of idle coroutines and dynamic scaling can be performed separately or in combination. The purpose is to match the distribution of coroutine resources with the current task load, taking into account both the high performance of local scheduling and the balance of global load.

[0035] Step 105: Return the coroutine that has completed the target task to the local cache pool for reuse.

[0036] In the embodiments of this application, after a coroutine completes its target task, it is not destroyed immediately, but returned to the local cache pool, waiting to be reused directly when the next task arrives.

[0037] Reuse refers to the process where, after a coroutine completes a task, its stack space, control block, and other resources are not released but are retained for subsequent tasks to continue using. This avoids the resource overhead caused by frequently creating and destroying coroutines, such as stack memory allocation and release, context initialization and destruction, etc.

[0038] Coroutines are prioritized to return to the local cache pool, allowing them to continue to be reused on their respective processor cores, further reducing cross-core migration and improving the local cache hit rate.

[0039] The coroutine pool scheduling method of this application achieves contention-free task distribution through a lock-free task queue, avoiding global lock contention; it uses a local cache pool to allow coroutines to be acquired and returned locally within the processor core, reducing the overhead of cross-processor migration and frequent creation and destruction; it binds task information to a dedicated coroutine pool to isolate different business resources and avoid resource contention; it ensures local scheduling efficiency while considering global resource utilization by controlling the acquisition order of the local cache pool, dedicated coroutine pool, and global coroutine pool; and it adjusts the distribution of coroutines or dynamically scales up and down based on the number of tasks and the number of local idle coroutines, allowing coroutine resources to adapt to load changes and avoiding task backlog or resource idleness. Therefore, this application can solve the problems of low scheduling efficiency, uneven resource allocation, and poor load adaptability under high concurrency.

[0040] In one possible implementation of this application embodiment, when constructing the global coroutine pool, the local cache pools corresponding to multiple processors respectively, and the lock-free task queue, the following methods can be used, but are not limited to: Based on the preset number of coroutines and the preset coroutine stack size, coroutine stack space and control block space are allocated from the memory pool. The control blocks of idle coroutines are linked together through an idle linked list. The global coroutine pool includes coroutine stack space and control block space. The coroutine stack space is used to store coroutines, and the control block space is used to store control blocks. The control blocks are used to control the coroutines to execute tasks. Create schedulers for each of the multiple processors, configure local cache pools for each scheduler, and set a first threshold, a second threshold, and a write-back threshold for each local cache pool. Distribute idle coroutines from the global coroutine pool to multiple local cache pools; Create a lock-free task queue that supports concurrent enqueueing and dequeueing of multiple producers and consumers, and divide the idle coroutines in the local cache pool into multiple groups of local wake-up queues.

[0041] In the embodiments of this application, the preset number of coroutines refers to the total number of coroutines that should be created according to the configuration parameters during the system initialization phase or the operation of the coroutine pool. It can be set according to the concurrency requirements of the business scenario. For example, for high-concurrency network request processing scenarios, a larger initial number of coroutines can be set to ensure task processing capacity.

[0042] The default coroutine stack size refers to the amount of stack space allocated to each coroutine. This stack space is used to store local variables, function call information, and return addresses during coroutine runtime. The coroutine stack size can be configured according to task complexity. For example, a smaller stack space can be used for coroutines performing simple calculations or lightweight tasks to save memory; a larger stack space can be used for tasks performing complex business logic or those with deep function call hierarchies to avoid stack overflow.

[0043] A memory pool refers to a large area of ​​memory that is pre-requested from the operating system. It is divided into multiple manageable memory blocks and used to uniformly allocate and reclaim the stack space, control block space, and queue node space required by coroutines.

[0044] The coroutine stack space is a memory area allocated from the memory pool to store the runtime stack data of coroutines. The control block space is a memory area allocated from the memory pool to store the control blocks of coroutines. The control block refers to the data structure used to record and manage coroutine metadata, which may include information such as coroutine identifier, coroutine state, priority, processor affinity mask, context pointer, local cache pool to which it belongs, and dedicated coroutine pool to which it belongs.

[0045] The control block is used to control the execution of tasks by coroutines. The scheduler can learn the current state of a coroutine and how to schedule it by reading and updating the information in the control block. The control blocks of idle coroutines are linked together by an idle linked list, which means that the control blocks of all coroutines that have not yet been assigned tasks are organized in a linked list structure, and the head of the linked list is stored in the management structure of the coroutine pool or local cache pool.

[0046] A free list is a data structure used to manage free resources. When allocating resources, a control block is taken from the head or tail of the list. When returning resources, the control block is reinserted into the list, thereby enabling fast acquisition and reclamation of coroutine resources.

[0047] A scheduler is a logical entity running on a processor that is responsible for retrieving tasks from a lock-free task queue, obtaining idle coroutines from a coroutine pool, and binding tasks and coroutines for execution. Each processor has one scheduler, which can run independently on that processor core to achieve parallel scheduling.

[0048] Configuring local cache pools for multiple schedulers means that each scheduler has its own dedicated local cache pool. The first threshold, second threshold, and write-back threshold are parameters used for managing the local cache pool's water level. The first threshold represents the upper limit of the number of idle coroutines in the local cache pool. When the number of idle coroutines exceeds this upper limit, it indicates resource redundancy in the local cache, and excess coroutines can be written back to the global coroutine pool. The second threshold represents the lower limit of the number of idle coroutines in the local cache pool. When the number of idle coroutines falls below this lower limit, it indicates insufficient reusable resources in the local cache, and coroutines need to be supplemented from the global coroutine pool. The write-back threshold refers to the minimum excess amount required to trigger a batch write-back operation. In other words, even if the number of idle coroutines in the local cache pool exceeds the first threshold, a write-back operation is only executed when the excess reaches the write-back threshold, thus avoiding frequent write-back triggers due to slight fluctuations in the number of idle coroutines.

[0049] The process of allocating idle coroutines from the global coroutine pool to multiple local cache pools can be completed during system initialization or dynamically during runtime based on load changes. During initialization, idle coroutines from the global coroutine pool can be evenly distributed to the local cache pools according to the number of processors or schedulers, ensuring that each local cache pool has a certain number of reusable coroutines in its initial state. For example, if the global coroutine pool pre-allocated a preset number of idle coroutines, and the system has multiple processors, these idle coroutines can be evenly distributed to the local cache pools, and the idle coroutine counts of each local cache pool and the global coroutine pool can be updated.

[0050] Multiple producers and multiple consumers means that this lock-free task queue allows multiple producer threads to write tasks to the queue at the same time, and also allows multiple consumer threads to read tasks from the queue at the same time, without relying on mutex locks or global locks to ensure data consistency throughout the entire concurrent operation process.

[0051] A lock-free task queue refers to a queue structure implemented using a lock-free concurrency control mechanism, capable of sorting or managing tasks according to their priority. When the scheduler acquires tasks, it can retrieve tasks from the corresponding lock-free task queue in descending order of priority, thereby ensuring that high-priority tasks obtain coroutine resources first.

[0052] Dividing idle coroutines in the local cache pool into multiple local wake-up queues is to enable targeted wake-up when a task arrives.

[0053] A partial wake-up queue is a queue formed by dividing idle coroutines according to a certain grouping rule. Each partial wake-up queue independently manages a portion of the idle coroutines. When a new task is assigned to the lock-free task queue, the scheduler can select some queues from multiple partial wake-up queues for wake-up operations based on task priority, without having to wake up all idle coroutines, thereby reducing concurrency contention and processor overhead caused by wake-up operations.

[0054] This application reduces the overhead of frequent system calls and lowers the risk of memory fragmentation by uniformly pre-allocating coroutine stack space and control block space through a memory pool; it achieves rapid allocation and reclamation of coroutine resources by linking control blocks through a free linked list; it enables coroutines to be reused locally by configuring an independent local cache pool for each processor, reducing cache consistency overhead caused by cross-processor migration; it provides a basis for subsequent adaptive water level management of local cache by setting a first threshold, a second threshold, and a write-back threshold; it eliminates the lock contention bottleneck in the task distribution process by creating a lock-free task queue that supports concurrent operations of multiple producers and consumers, improving the efficiency of task enqueueing and dequeueing in high-concurrency scenarios; and it lays the foundation for subsequent priority-based targeted wake-up, reduced wake-up latency, and reduced invalid contention by dividing idle coroutines into multiple groups of local wake-up queues.

[0055] In one possible implementation of this application embodiment, the lock-free task queue includes a first priority queue, a second priority queue, and a third priority queue, wherein tasks in the first priority queue are executed first; The lock-free task queue uses a circular array and supports concurrent enqueueing by multiple producers and concurrent enqueueing and dequeueing by multiple consumers through atomic operations.

[0056] In the embodiments of this application, a first priority queue is used to store high-priority tasks, a second priority queue is used to store medium-priority tasks, and a third priority queue is used to store low-priority tasks. Priority refers to the execution priority specified in the task information during the task submission phase, reflecting the task's requirements for real-time performance or importance.

[0057] When the scheduler retrieves tasks from the lock-free task queue, it attempts to retrieve them sequentially according to the order of the first priority queue, the second priority queue, and the third priority queue. Therefore, tasks in the first priority queue are executed first, ensuring that tasks with high real-time requirements or high importance can obtain coroutine resources and be put into execution first.

[0058] A circular array is an array structure where the first and last elements are connected, with a fixed or configurable capacity. The array returns to the first element after the last element, forming a logical loop. A circular array uses head and tail pointers to identify the current enqueue and dequeue positions. When a task needs to be enqueued, the producer thread atomically reserves a slot at the position pointed to by the tail pointer, writes the task pointer to that slot, and then updates the tail pointer. When a task needs to be dequeued, the consumer thread atomically reads the task pointer from the position pointed to by the head pointer and then updates the head pointer.

[0059] Atomic operations are operations that cannot be interrupted by other threads during execution. Through atomic operations, multiple threads can safely access shared variables in a queue concurrently without using mutexes or global locks. Producers are threads or scheduling entities that submit tasks to the queue; they compete for the slot pointed to by the tail pointer through atomic operations, thus achieving concurrent enqueueing. Consumers are threads or scheduling entities that retrieve tasks from the queue; they compete for the slot pointed to by the head pointer through atomic operations, thus achieving concurrent dequeueing.

[0060] This application ensures that high-priority tasks have priority in execution by dividing the queue into priority queues, thus reducing end-to-end latency for high real-time tasks. It also improves memory locality of task storage through a circular array structure, reducing the overhead of dynamic memory allocation. Furthermore, it implements lock-free concurrency control through atomic operations, eliminating global lock contention and enabling multiple producers and consumers to access the queue simultaneously and efficiently. This improves task distribution and retrieval throughput in high-concurrency scenarios and reduces scheduling latency.

[0061] In one possible implementation of this application embodiment, before determining the dedicated coroutine pool associated with the target task based on the task information of the target task, the following methods may also be used, but are not limited to: allocating a task structure for the target task from the memory pool, filling the target task with task function pointers, parameters, priorities, and coroutine pool binding fields in the task structure; and determining the coroutine pool pointed to by the coroutine pool binding fields in the task structure as a dedicated coroutine pool.

[0062] In the embodiments of this application, the task structure refers to a data structure used to carry metadata and execution information related to the target task. As the basic management unit of the task in the scheduling process, it records various information required by the scheduler to execute the task.

[0063] A task function pointer is a pointer to the function or code entry point that the target task actually needs to execute. The scheduler can call and execute the business logic corresponding to the target task through this task function pointer. Parameters refer to the input data or configuration information required when the target task is executed. These can include various forms such as numbers, strings, object references, and callback function pointers, and their specific content is determined by the business requirements of the target task itself. Priority refers to the priority of the target task during scheduling and execution. It indicates which lock-free task queue the target task should be assigned to, and its priority order when acquiring coroutine resources. The coroutine pool binding field is a field in the task structure used to specify the dedicated coroutine pool associated with the target task, enabling the scheduler to quickly locate the dedicated coroutine pool to which the target task is bound when acquiring coroutine resources.

[0064] When submitting a target task, the task submitter can explicitly specify which dedicated coroutine pool's resources the target task should use by setting the coroutine pool binding field in the task structure. The coroutine pool pointed to by this coroutine pool binding field is determined as the dedicated coroutine pool associated with the target task. Through explicit binding, tasks from different services can be bound to their respective dedicated coroutine pools, thus prioritizing the acquisition of idle coroutines from the bound pool during scheduling and execution, without consuming resources from other service pools. Furthermore, by setting the coroutine pool binding field in the task structure, the scheduler can directly read the binding information from the task structure without additionally querying the global mapping table or relying on external configuration when processing tasks.

[0065] This application reduces memory allocation overhead and memory fragmentation during task submission by allocating task structures from a memory pool. By filling the task structure with task function pointers, parameters, priorities, and coroutine pool binding fields, the target task completes full metadata encapsulation at the submission stage, eliminating the need for the scheduler to repeatedly query or infer task attributes during subsequent scheduling, thus improving scheduling efficiency. By defining the coroutine pool pointed to by the coroutine pool binding field as a dedicated coroutine pool, dynamic binding between tasks and coroutine pools is achieved, enabling tasks from different businesses, tenants, or with different service quality requirements to use independent coroutine resource pools, thereby achieving resource isolation, fault isolation, and service quality assurance.

[0066] In one possible implementation of this application embodiment, when distributing the target task to the corresponding lock-free task queue according to the preset task priority, and obtaining the idle coroutine in the order of local cache pool, dedicated coroutine pool, and global coroutine pool, the following methods can be used, but are not limited to: extracting the idle coroutine from the idle linked list of the target local cache pool corresponding to the target scheduler, where the target scheduler is the scheduler that receives the processing request of the target task. In response to the absence of idle coroutines in the target local cache pool, an idle coroutine is obtained from the idle list of the dedicated coroutine pool based on the task information of the target task. In response to the lack of free coroutines in the dedicated coroutine pool, a free coroutine is obtained from the free list of the global coroutine pool. Change the status of the acquired idle coroutine from idle to running, and record the scheduler to which the idle coroutine currently belongs.

[0067] In the embodiments of this application, the target scheduler refers to the scheduler entity currently responsible for processing the target task. The target scheduler first attempts to obtain idle coroutines from its corresponding target local cache pool. The target local cache pool refers to a local coroutine cache structure exclusive to the target scheduler. The idle list is a data structure that links idle coroutine control blocks together in a linked list format. The head of the list is stored in the management information of the local cache pool, and the scheduler can quickly extract or insert the control block of an idle coroutine by manipulating the head of the list.

[0068] The target scheduler retrieves idle coroutines from the idle list of the target's local cache pool. This operation is equivalent to prioritizing the use of coroutine resources that are already cached locally on the current processor core.

[0069] When there are no idle coroutines available in the target local cache pool, it means that the reusable coroutine resources of the current processor core have been exhausted. At this time, the target scheduler turns to the dedicated coroutine pool bound to the target task and obtains an idle coroutine from the idle list of the dedicated coroutine pool.

[0070] When neither the target local cache pool nor the dedicated coroutine pool bound to the target task can provide idle coroutines, it indicates that both local and dedicated resources are in a state of strain. At this point, the target scheduler retrieves idle coroutines from the idle list of the global coroutine pool to ensure that the target task can still obtain coroutine resources and be executed. Idle coroutines in the global coroutine pool may come from pre-allocation during system initialization, or they may come from idle coroutines written back from other local cache pools or dedicated coroutine pools during resource redundancy.

[0071] "Running" refers to the state where a coroutine is currently executing a task or has been selected by the scheduler and is about to execute a task. Changing the coroutine's state from idle to running prevents the same idle coroutine from being repeatedly acquired by multiple schedulers, ensuring consistency in coroutine resource allocation.

[0072] Meanwhile, the target scheduler records the scheduler to which the idle coroutine currently belongs, which means that the coroutine has established an affiliation with the target scheduler during its use. After the coroutine completes the target task, it can return to the local cache pool corresponding to the target scheduler first based on the affiliation, thereby realizing the closed-loop reuse of coroutines.

[0073] This application prioritizes retrieving idle coroutines from the target local cache pool, enabling coroutine reuse on the current processor core, reducing cross-core migration, improving cache hit rate, and lowering context switching overhead. Secondly, when the local cache is insufficient, it prioritizes retrieving idle coroutines from the task-bound dedicated coroutine pool, achieving resource and fault isolation between services and ensuring the service quality of core or specific services. Thirdly, when no idle coroutines are available in the dedicated coroutine pool, it retrieves idle coroutines from the global coroutine pool, ensuring the system can maintain task processing capacity even when local resources are scarce.

[0074] In one possible implementation of this application embodiment, when adjusting the distribution of idle coroutines between the local cache pool and the global coroutine pool based on the number of tasks in the lock-free task queue and the number of idle coroutines in the local cache pool, the following methods can be used, but are not limited to: The number of idle coroutines in the local cache pool is detected within a preset statistical period, and the preset statistical period in which the number of idle coroutines is greater than a first threshold is determined as the first statistical period. In response to the first statistical period reaching a first preset threshold number of consecutive times and the target number of coroutines reaching a write-back threshold, the first number of idle coroutines in the local cache pool are written back to the global coroutine pool. The target number of coroutines is the sum of the differences between the number of idle coroutines and the first threshold in the first consecutive statistical period. The preset statistical period when the number of idle coroutines is less than the first threshold is determined as the second statistical period; In response to the number of consecutive occurrences in the second statistical period reaching the first preset threshold, a second number of idle coroutines are extracted from the global coroutine pool and added to the local cache pool.

[0075] In the embodiments of this application, the preset statistical period refers to the time unit for the load monitoring module or scheduler to collect load data at preset time intervals. For example, after each preset statistical period, the system reads load indicators such as the number of idle coroutines in the local cache pool and the number of tasks in the lock-free task queue.

[0076] Within each preset statistical period, the number of idle coroutines in the local cache pool is detected and compared with a first threshold. When the number of idle coroutines in the local cache pool exceeds the first threshold within a certain preset statistical period, it indicates that the number of idle coroutines cached in the local cache pool has exceeded the upper limit of the normal range. At this time, the preset statistical period is determined as the first statistical period. The first statistical period characterizes the time segment in which the local cache pool is in an overloaded state.

[0077] After determining the first statistical period, the system further determines whether the number of consecutive occurrences in the first statistical period reaches the first preset number threshold, and whether the number of target coroutines reaches the write-back threshold.

[0078] The first preset number threshold refers to the number of consecutive cycles used to determine whether the overload state continues. Only when the number of idle coroutines in the local cache pool is greater than the first threshold for multiple consecutive preset statistical cycles, and the number of consecutive cycles reaches the first preset number threshold, is the local cache pool considered to be in a continuous overload state, rather than an occasional resource redundancy.

[0079] The target number of coroutines reflects the cumulative excess of the number of idle coroutines in the local cache pool beyond the first threshold during a period of sustained overload. When the target number of coroutines reaches the write-back threshold, it indicates that the redundancy of idle coroutines in the local cache pool is sufficiently high, and the excess idle coroutines need to be transferred to the global coroutine pool to reduce the burden on the local cache pool and supplement global resources.

[0080] The first quantity refers to the number of idle coroutines involved in this write-back operation. It can be determined based on the target number of coroutines. For example, the first quantity can be set to be equal to the target number of coroutines, or set to a certain percentage of the target number of coroutines.

[0081] On the other hand, a preset statistical period in which the number of idle coroutines is less than a first threshold is defined as the second statistical period. The second statistical period represents the time segment in which the local cache pool is in a low-load state. After determining the second statistical period, it is determined whether the number of consecutive occurrences of the second statistical period reaches a first preset threshold.

[0082] When the number of consecutive occurrences in the second statistical period reaches the first preset threshold, it indicates that the local cache pool is continuously underloaded. At this time, a second number of idle coroutines are extracted from the global coroutine pool and added to the local cache pool. The second number refers to the number of idle coroutines involved in this replenishment operation, which can be determined based on the difference between the first threshold and the number of idle coroutines in multiple consecutive second statistical periods. For example, the second number can be set as the sum of the differences between the first threshold and the number of idle coroutines in consecutive second statistical periods, or as a certain proportion of the sum of these differences.

[0083] It should be noted that both write-back and supplement operations use batch processing. Batch processing means transferring multiple idle coroutines simultaneously in a single operation, rather than transferring them one by one.

[0084] This application can promptly identify whether the local cache pool is overloaded or underloaded by detecting the number of idle coroutines in the local cache pool within a preset statistical period and comparing it with a first threshold. By judging for multiple consecutive periods and setting a write-back threshold, it avoids frequent write-back or replenishment operations triggered by short-term load fluctuations, thus improving system stability. By writing back a first number of idle coroutines to the global coroutine pool in batches when the local cache pool is overloaded, it can reduce the local load of the local cache pool, avoid local resource redundancy, and replenish resources for the global coroutine pool, which is beneficial to global load balancing. By extracting a second number of idle coroutines from the global coroutine pool in batches and adding them to the local cache pool when the local cache pool is underloaded, it can promptly replenish local reusable coroutine resources and ensure that local scheduling capabilities do not decline. Through batch write-back and batch replenishment, it reduces the number of global resource accesses and concurrent contention, and improves the efficiency of coroutine resource adjustment.

[0085] In one possible implementation of this application embodiment, the first quantity is the sum of the differences between the number of idle coroutines and the first threshold in a continuous first statistical period; The second quantity is the sum of the differences between the first threshold and the number of idle coroutines in a consecutive second statistical period.

[0086] In one possible implementation of this application embodiment, when dynamically scaling up and down the global coroutine pool, it can be implemented in the following ways, but not limited to: in response to the number of consecutive times in a preset statistical period where the number of tasks is greater than a preset task number threshold and the proportion of idle coroutines is less than a preset proportion threshold, new idle coroutines are allocated from the memory pool to the global coroutine pool and / or the local cache pool according to the first expansion step size, and the proportion of idle coroutines is the proportion of idle coroutines in the local cache pool to all coroutines; In response to the fact that the number of idle coroutines in both the global coroutine pool and the local cache pool is zero and the number of tasks is greater than the preset task number threshold, new idle coroutines are allocated from the memory pool to the global coroutine pool and / or the local cache pool according to the second expansion step size, and the second expansion step size is greater than the first expansion step size. In response to the number of consecutive times in a preset statistical period where the number of tasks is less than a preset task number threshold and the proportion of idle coroutines is greater than a preset proportion threshold, the idle coroutines in the global coroutine pool are destroyed according to a preset shrinkage ratio, while retaining the minimum number of coroutines.

[0087] In the embodiments of this application, the preset task quantity threshold is a pre-set reference number of tasks used to determine whether the current backlog of tasks has reached a point where additional coroutine resources are needed. The idle coroutine ratio refers to the proportion of idle coroutines in the local cache pool to all coroutines in the local cache pool. When the idle coroutine ratio is low, it indicates that locally available reusable idle coroutine resources are relatively scarce. The preset ratio threshold is a pre-set reference ratio used to determine whether the idle coroutine ratio is too low and requires additional coroutine resources.

[0088] The second preset threshold is a threshold for the number of consecutive cycles used to determine whether a high-load state persists. Only when the number of tasks continuously exceeds the preset task number threshold, the proportion of idle coroutines continuously falls below the preset proportion threshold, and the number of consecutive statistical cycles of the above-mentioned state reaches the second preset threshold, is it determined that the current state is indeed in a state of continuous high load, and regular capacity expansion is required.

[0089] The first expansion step size refers to the number of new idle coroutines created in a single operation during normal expansion. It can be set according to the system resource status and load growth trend. Adding new idle coroutines to the global coroutine pool can increase the global resource buffer, which can be obtained by various local cache pools or dedicated coroutine pools when needed. Adding new idle coroutines to the local cache pool can directly increase the local reusable coroutine resources and improve the efficiency of the local scheduler in obtaining idle coroutines.

[0090] In emergency expansion, both the global coroutine pool and the local cache pool have zero idle coroutines, indicating that all available idle coroutine resources in the system have been exhausted, and there are no idle coroutines available for the scheduler to acquire. Simultaneously, the number of tasks still exceeds the preset task threshold, indicating that tasks are still piling up and face the risk of not being able to obtain coroutine resources for execution in a timely manner. In this emergency situation, instead of waiting for the results of multiple consecutive statistical periods, emergency expansion is triggered immediately. The second expansion step size is the number of new idle coroutines created in a single operation during emergency expansion, which is greater than the first expansion step size used in regular expansion.

[0091] Regarding scaling down, if the number of tasks is less than a preset task threshold, it indicates that there are relatively few tasks to be processed; if the proportion of idle coroutines is greater than a preset proportion threshold, it indicates that there are relatively abundant reusable idle coroutine resources in the local cache pool, or even redundancy. When the preset number of consecutive occurrences of the above low-load state reaches a second preset threshold, it is determined that the current state is indeed continuously low-load, and a scaling down operation can be performed to reduce coroutine resource consumption.

[0092] The preset shrinkage ratio refers to the proportion of idle coroutines destroyed in a single shrinkage operation relative to the total number of coroutines that can be reduced. It can be set according to the system's requirements for resource reclamation speed and stability. According to the preset shrinkage ratio, idle coroutines are selected from the global coroutine pool for destruction. The destruction process includes removing the idle coroutine from the idle list, releasing its stack space and control block space back to the memory pool, and updating the total coroutine count.

[0093] The minimum number of coroutines is a pre-set lower limit on the number of coroutines that should be retained after scaling down.

[0094] It should be noted that regular expansion, emergency expansion, and shrinkage operations are all triggered based on the judgment results of multiple consecutive preset statistical periods. Regular expansion and shrinkage require a second preset threshold number of consecutive occurrences, effectively preventing frequent changes in the number of coroutines due to instantaneous fluctuations in task load. Emergency expansion, on the other hand, is triggered immediately in the extreme case where idle coroutine resources are completely exhausted and tasks are still piling up, ensuring the system can quickly restore its task processing capacity.

[0095] This application uses a combination of task quantity and idle coroutine ratio to determine whether to expand or shrink the system, allowing adjustments to the number of coroutines to more accurately reflect the actual system load and avoiding misjudgments caused by a single indicator. By using judgments over multiple preset statistical periods and setting a second preset threshold, it avoids frequent expansion or shrinkage triggered by short-term load fluctuations, improving system stability. By triggering emergency expansion when idle coroutine resources are exhausted and tasks are still backlogged, and using a second expansion step larger than the first expansion step, it can quickly alleviate the emergency situation of resource depletion and task backlog, ensuring the system's task processing capacity under extreme loads. By destroying idle coroutines according to a preset shrinkage ratio and retaining the minimum number of coroutines during sustained low loads, it can reclaim excess resources, reduce memory usage, and avoid excessive shrinkage and frequent jitter.

[0096] In one possible implementation of this application embodiment, when returning a coroutine that has completed the target task to a local cache pool for reuse, it can be implemented in, but is not limited to, the following ways: In response to a coroutine that has completed the target task being a coroutine in its local cache pool, the coroutine that has completed the target task is written back to its local cache pool. If a coroutine that has completed the target task is not a coroutine in its local cache pool, the coroutine that has completed the target task is written to the remote return queue. Based on the remote return queue, the coroutine that has completed the target task is written back to the dedicated coroutine pool or the global coroutine pool.

[0097] In the embodiments of this application, the local cache pool refers to the local cache pool to which the coroutine belonged before executing the target task, typically the local cache pool associated with the processor core corresponding to the scheduler executing the coroutine. When the coroutine is obtained from the target scheduler's own local cache pool and used to execute the target task, the coroutine still belongs to that local cache pool after the current execution ends. In this case, the coroutine that has completed executing the target task can be directly written back to its local cache pool.

[0098] Write-back refers to re-inserting the coroutine's control block into the free list of its local cache pool and updating the free coroutine count and status information of that local cache pool. Since the scheduler executing the coroutine is itself the owner of its local cache pool, this write-back operation can be completed directly without locking, representing a fast path in the return operation. Through this fast path, the coroutine can quickly return to its original local cache pool after completing its task.

[0099] In actual scheduling, when the target scheduler has no free coroutines in its local cache pool, it obtains an free coroutine from the dedicated coroutine pool bound to the target task, or from the global coroutine pool, and uses that coroutine to execute the target task. At this point, the coroutine that has completed the target task does not belong to the target scheduler's own local cache pool, but rather to the dedicated coroutine pool or the global coroutine pool. In this situation, if the target scheduler directly writes the coroutine back to the free list of its original pool, it may cause concurrent conflicts with the pool's owner thread or other schedulers, because the original pool's free list may be accessed simultaneously by its owner thread or other threads. To avoid this concurrent conflict, a remote return mechanism is used.

[0100] The remote return queue is a queue set up in the coroutine pool to temporarily store coroutine control blocks returned by non-owner threads. When a coroutine that has completed the target task is not a coroutine in its local cache pool, the target scheduler writes the coroutine's control block into the remote return queue of the coroutine's original pool.

[0101] The owner thread is the thread that created or owns the coroutine pool, typically the scheduler thread or business thread that created it. Before acquiring an idle coroutine from its own coroutine pool, the owner thread checks if the remote return queue of the coroutine pool is empty. If there are pending coroutine control blocks in the remote return queue, the owner thread will transfer these coroutine control blocks in batches to the idle list of the coroutine pool and update the idle coroutine count.

[0102] It should be noted that when a coroutine is returned, its status needs to be changed from running or runnable to idle, and the association information related to the coroutine and this execution needs to be cleared or updated.

[0103] This application enables coroutines belonging to their local cache pool to be written directly back to their local cache pool via a fast path without locking, thus achieving the nearest reuse of coroutines within the processor core. For coroutines not belonging to the local cache pool, they are returned through a remote return queue, avoiding concurrency conflicts caused by non-owner threads directly operating on the free lists of other pools, and ensuring the safety and consistency of the coroutine return process.

[0104] In one possible implementation of this application embodiment, when retrieving an idle coroutine from the idle linked list of the target local cache pool corresponding to the target scheduler, it can be done in the following ways, but is not limited to: Idle coroutines are divided into multiple local wake-up queues based on their identifier hashes, and atomic flags are set for each local wake-up queue. The queue to be woken up is determined from multiple local wake-up queues according to the preset task priority, and the atomic flag position of the queue to be woken up is marked as to be woken up through atomic swap; In response to the existence of an idle coroutine in the wake-up queue, the idle coroutine is extracted from the wake-up queue, and the status of the extracted idle coroutine is changed from idle to runnable.

[0105] In the embodiments of this application, when the target scheduler needs to obtain an idle coroutine from the target local cache pool, it does not directly perform a unified search or unified wake-up of all idle coroutines in the local cache pool. Instead, it pre-divides the idle coroutines in the local cache pool according to their identifiers using hashing to obtain multiple groups of local wake-up queues.

[0106] An identifier is a unique identifier used to distinguish different coroutines, such as a coroutine number, a coroutine control block address, or a sequence number assigned during coroutine creation. A local wake-up queue is a queue structure maintained independently for each group of idle coroutines. Each local wake-up queue stores the control blocks of the idle coroutines belonging to that group, and the local wake-up queues are independent of each other, allowing for separate management, wake-up, and retrieval operations.

[0107] After dividing the system into multiple local wake-up queues, atomic flags are set for each queue. An atomic flag is a status flag corresponding to each local wake-up queue that can be read and modified atomically. It indicates whether the queue needs to be woken up or is already in a waiting state. Because reading and modifying atomic flags are done atomically, multiple schedulers or wakers can concurrently access the atomic flags of the same local wake-up queue without data races or inconsistent states. Setting atomic flags allows for quick determination of whether a particular local wake-up queue has already been selected by another waker, thus avoiding duplicate and invalid wake-ups.

[0108] Tasks with different priorities have varying degrees of urgency in waking up coroutines. Therefore, the system selects some or all of the multiple local wake-up queues as the wake-up queue based on preset task priorities. For example, for higher-priority tasks, more local wake-up queues can be selected as the wake-up queue to obtain idle coroutines more quickly; for lower-priority tasks, fewer local wake-up queues can be selected as the wake-up queue to save on the overhead of wake-up operations.

[0109] The wake-up queue refers to the local wake-up queue targeted by this wake-up operation. Once selected, it needs to be further determined whether there are any idle coroutines within it, and if so, a retrieval operation is performed. After determining the wake-up queue, atomic swaps are used to mark the atomic flags of the wake-up queue as ready to be woken up.

[0110] After successfully marking the atomic flag of the wake-up queue as pending wake-up via atomic swap, the system further checks whether there are idle coroutines in the queue. If idle coroutines exist, one or more control blocks of these idle coroutines are extracted, and their states are changed from idle to runnable. Runnable means that the coroutine has been woken up and has the conditions to execute a task, but has not yet been actually put into operation by the scheduler. Changing the coroutine's state from idle to runnable prevents the coroutine from being repeatedly extracted by other wakers or the scheduler, ensuring the consistency of coroutine state transitions.

[0111] If there are no idle coroutines in the wake-up queue, it means that there are currently no available idle coroutines in this local wake-up queue. In this case, the wake-up queue can be skipped, or its atomic flag can be restored to its initial state so that the wake-up can be triggered again when a subsequent task arrives.

[0112] This application divides idle coroutines into multiple local wake-up queues based on their identifier hashes, enabling wake-up operations to be performed on a group-by-group basis. This avoids global unified wake-up or global polling, thereby significantly reducing the scope and intensity of concurrency contention. The wake-up queues are determined according to preset task priorities, allowing high-priority tasks to trigger wake-ups in more groups, thus acquiring idle coroutine resources more quickly and meeting high real-time requirements. Conversely, low-priority tasks trigger wake-ups in fewer groups, saving on wake-up operation overhead and achieving a match between wake-up intensity and task priority.

[0113] In one possible implementation of this application embodiment, when determining the wake-up queue from multiple groups of local wake-up queues according to a preset task priority, it can be implemented in, but is not limited to, the following ways: In response to the target task being in the first priority queue of the lock-free task queue, the first group of local wake-up queues in the multiple groups of local wake-up queues is determined as the wake-up queue; In response to the target task being in the second priority queue of the lock-free task queue, the second group of local wake-up queues in the multiple groups of local wake-up queues is determined as the wake-up queue; In response to the target task being in the third priority queue of the lock-free task queue, the third group of local wake-up queues in the multiple groups of local wake-up queues is determined as the wake-up queue; Among them, the number of the first group is greater than the number of the second group, which is greater than the number of the third group, and there are no idle coroutines in different wake-up queues.

[0114] In the embodiments of this application, the scheduler attempts to acquire tasks sequentially according to the first priority queue, the second priority queue, and the third priority queue. Therefore, tasks in the first priority queue have a higher execution priority than tasks in the second and third priority queues. The first group number refers to the number of local wake-up queue groups selected for tasks in the first priority queue. Its value is relatively large so that idle coroutines can be woken up in a wider range, thereby quickly acquiring available coroutine resources.

[0115] When a target task is in the second priority queue, it indicates that the target task is a medium-priority task, and its real-time execution requirements are between those of high-priority and low-priority tasks. The second group number refers to the number of local wake-up queues selected for tasks in the second priority queue. The second group number is less than the first group number; that is, for medium-priority tasks, the system selects fewer wake-up queues than for high-priority tasks.

[0116] When a target task is in the third priority queue, it indicates that the task is a low-priority task with relatively low real-time requirements. It can typically be processed when system resources are relatively abundant. The number of third groups refers to the number of local wake-up queues selected for tasks in the third priority queue. The number of third groups is less than the number of second groups, meaning that for low-priority tasks, the number of wake-up queues selected by the system is further reduced. By selecting a smaller number of local wake-up queues, low-priority tasks only trigger idle coroutine wake-ups within a small range, thus saving processor overhead and concurrency contention caused by wake-up operations, and preventing low-priority tasks from excessively consuming system resources and affecting the execution of high-priority tasks.

[0117] Furthermore, during the process of determining the wake-up queues, no two idle coroutines exist in the same queue. Since multiple local wake-up queues are obtained by partitioning idle coroutines according to their identifier hashes, each idle coroutine belongs to only one local wake-up queue. Therefore, the idle coroutines contained in different local wake-up queues do not overlap. This avoids abnormal situations such as duplicate wake-ups, resource contention, and inconsistent states.

[0118] This application achieves adaptive matching between wake-up intensity and task priority by selecting local wake-up queues of the first, second, or third group size as wake-up queues based on the priority queue level of the target task. For high-priority tasks, selecting a larger number of wake-up queues can quickly wake up a sufficient number of idle coroutines, reducing the waiting latency of high-priority tasks. By ensuring that the same idle coroutine does not exist in different wake-up queues, the duplicate selection or extraction of the same idle coroutine is avoided, preventing duplicate wake-ups and resource contention, and improving the correctness and efficiency of coroutine wake-up and extraction.

[0119] In one possible implementation of the embodiments of this application, the following methods may also be used, but are not limited to: The number of tasks, the proportion of idle coroutines, the first threshold and the second threshold are collected according to the preset statistical period to obtain the collection results, and the collection results are saved to the circular history buffer. The first threshold, second threshold, preset task quantity threshold, and preset ratio threshold are dynamically adjusted based on the load change trend in the cyclic historical buffer.

[0120] In the embodiments of this application, a circular history buffer refers to a buffer structure with a fixed storage capacity that is written to and overwritten in a circular manner. Internally, it can organize multiple storage units in the form of an array or a linked list, with each storage unit used to store the collection results corresponding to a preset statistical period.

[0121] By using a circular history buffer, load change records over a period of time can be saved with minimal storage overhead, without requiring unlimited storage growth. The saved collection results can include the number of tasks in the current period, the proportion of idle coroutines, the first threshold, and the second threshold, as well as intermediate indicators calculated from these data, such as the deviation of the number of tasks from a preset task number threshold, the deviation of the proportion of idle coroutines from a preset proportion threshold, and the deviation of the number of idle coroutines in the local cache pool from the first or second threshold.

[0122] Load change trend refers to the direction and magnitude of changes in the number of tasks and the proportion of idle coroutines, as reflected in the data collected over multiple consecutive preset statistical periods in the circular history buffer. For example, if the number of tasks in the circular history buffer shows a gradual upward trend over multiple consecutive periods, while the proportion of idle coroutines shows a gradual downward trend, it indicates that the system load is continuously increasing, the risk of task backlog is increasing, and local reusable coroutine resources are becoming strained. By analyzing the load change trend in the circular history buffer, it is possible to determine whether the current system is in a load increase or decrease phase, and the severity of the load change. Based on this, the first threshold, the second threshold, the preset task number threshold, and the preset proportion threshold are dynamically adjusted.

[0123] Specifically, when the load trend indicates that the system load is continuously increasing, the system can appropriately lower the first threshold or the preset task number threshold, so that the write-back operation or expansion operation of the local cache pool is triggered earlier. When the load trend indicates that the system load is continuously decreasing, the system can appropriately raise the first threshold or the preset task number threshold, making the triggering conditions for write-back and expansion operations more stringent, and avoiding frequent triggering of resource adjustment operations during the load reduction process.

[0124] This application collects the number of tasks, the proportion of idle coroutines, a first threshold, and a second threshold at a preset statistical period, and saves the collection results to a circular history buffer. This allows for continuous recording of load changes with minimal storage overhead, providing a data foundation for subsequent trend analysis and threshold adjustment. By dynamically adjusting the first threshold, the second threshold, the preset task number threshold, and the preset proportion threshold based on the load change trend in the circular history buffer, the resource adjustment strategy can adaptively change with load changes, avoiding the response lag or frequent triggering issues that may occur with fixed thresholds in load fluctuation scenarios.

[0125] In one possible implementation of the embodiments of this application, the following methods may also be used, but are not limited to: Monitor memory pool usage; An alarm event is generated in response to the usage rate exceeding a preset usage rate threshold.

[0126] In the embodiments of this application, utilization rate refers to the proportion of memory capacity currently allocated in the memory pool to the total capacity of the memory pool, or the proportion of the number of memory blocks currently in use in the memory pool to the total number of memory blocks. Utilization rate reflects the sufficiency of free memory resources available for subsequent allocation in the memory pool. When the utilization rate is low, it indicates that there is still a lot of free memory available for allocation in the memory pool; when the utilization rate is high, it indicates that the free memory in the memory pool is relatively scarce.

[0127] Alarm events are event information used to notify upper-layer systems, schedulers, or operations and maintenance modules of the high resource pressure in the memory pool. Alarm events may include information such as the current memory pool utilization rate, remaining available memory, total memory pool capacity, trigger time, and possible suggested handling measures. Alarm events can be generated by sending notification messages to pre-registered alarm receivers, recording alarm entries in the system log, or setting alarm flags in shared memory for other modules to query.

[0128] This application monitors the memory pool usage rate, enabling the system to grasp the remaining free memory in the memory pool in real time or periodically, and promptly detect situations where memory resources are becoming strained. By setting a preset usage rate threshold and generating an alarm event when the usage rate exceeds the threshold, it can issue a warning before the memory pool is completely exhausted, providing a buffer time for upper-layer modules to take countermeasures.

[0129] In one possible implementation of the embodiments of this application, the following methods may also be used, but are not limited to: In response to a blocking operation during the execution of the target task by the coroutine, the coroutine's state is changed from running to blocked, and another idle coroutine is retrieved from the local cache pool or the global coroutine pool to execute the target task. After a blocked coroutine completes its wait, the blocked coroutine is put back into the local cache pool.

[0130] In the embodiments of this application, blocking operation refers to an operation in which a coroutine needs to wait for a certain external condition to be met or a certain resource to become available before it can continue to execute, such as input / output waiting, network data reception waiting, etc.

[0131] When the scheduler or the coroutine itself detects that the currently executing target task has entered a blocking operation, it changes the coroutine's state from running to blocked. After changing the coroutine's state to blocked, the scheduler can retrieve another idle coroutine from the local cache pool or the global coroutine pool to execute the target task.

[0132] When the condition for a blocking operation to wait for is met, or the resource it is waiting for becomes available, the previously blocked coroutine completes its wait and is ready to continue execution. At this point, the blocked coroutine is put back into the local cache pool. Putting it back in means reinserting the control block of the completed coroutine into the free list of the local cache pool, changing the coroutine's state from blocked to idle or runnable, and updating the idle coroutine count in the local cache pool.

[0133] This application detects blocking operations during the execution of a target task by a coroutine and changes the coroutine's state from running to blocked, thereby releasing processor resources in a timely manner and preventing the processor from idling while waiting for blocking conditions. By retrieving another idle coroutine from the local cache pool or the global coroutine pool to execute the target task, the processing of the target task can continue during the original coroutine's blocking period, improving processor utilization and task processing efficiency.

[0134] In one possible implementation of this application, to facilitate understanding of the implementation process, an example is provided as follows: like Figure 2 As shown in the figure, this application embodiment also provides a schematic diagram of the overall framework of a coroutine pool scheduling system, which consists of eight collaborative modules: Lock-free multi-queue and local cache scheduling module: The system adopts a lock-free multi-producer multi-consumer (MPMC) task queue at the global level, supporting multiple task producers to add tasks to the queue simultaneously, and multiple schedulers to retrieve tasks from the queue simultaneously. The entire process requires no locking and there is no lock contention. Simultaneously, each Central Processing Unit (CPU) core has its own independent local cache pool to store coroutines that have just completed their tasks and are in an idle state. When a coroutine completes its task, it will first return to the local cache pool of the current core, achieving "nearest reuse within the core" and avoiding cross-core migration. The scheduler follows a three-level resource acquisition order when acquiring coroutines: first, it takes resources from the local cache pool; if there are no free resources locally, it requests resources from the dedicated coroutine pool bound to the task; if the bound pool is also full, it finally allocates resources from the global coroutine pool.

[0135] The task and coroutine pool dynamic binding module allows explicit specification of a dedicated coroutine pool during the task submission phase via the coroutine pool binding field in the task information. This supports independent resource allocation for multiple services, multiple tenants, and multiple priorities. During scheduling and execution, resources are preferentially allocated from the bound pool. Resources, queues, and scheduling between different service pools are completely independent, achieving fault domain isolation and effectively ensuring the service quality of core services.

[0136] The adaptive cache pool water level management module configures three key parameters for each local cache pool: high water level, low water level, and write-back threshold. The system continuously monitors the number of local coroutines. When the number of coroutines exceeds the high water level for several consecutive statistical periods (e.g., three consecutive periods), and the number exceeding the high water level reaches or exceeds the write-back threshold, an overload write-back is triggered. The excess coroutines are written back to the global coroutine pool in batches, thereby reducing the local load on the current CPU core. Conversely, when the number of local coroutines falls below the low water level for several consecutive periods, the system will batch-pull idle coroutines from the global coroutine pool to supplement them, ensuring that scheduling capabilities do not decrease. All write-back and supplementation operations are processed in batches, reducing synchronous access to global resources. This ensures high performance for each core while also maintaining load balancing across the entire system.

[0137] The dynamic scaling management module uses task queue length and the proportion of idle coroutines to comprehensively determine whether to expand or shrink the system. For expansion, when the queue length exceeds a threshold and there are insufficient idle coroutines, new coroutines are created in a set step size; if there are no idle coroutines globally, emergency expansion is triggered (in this case, the expansion step size is doubled, and new coroutines are immediately allocated from the memory pool). For shrinking, when the proportion of idle coroutines is too high and the queue remains empty, idle coroutines are gradually destroyed proportionally, but the initially set minimum number of coroutines is retained to avoid frequent fluctuations. All scaling decisions are based on statistical judgments over multiple consecutive periods to prevent frequent resource changes due to short-term load fluctuations.

[0138] The hierarchical task scheduling module maintains high, medium, and low priority queues to ensure that high-priority tasks get coroutines first. It supports batch task retrieval (configurable to retrieve 1 to 3 tasks at a time), reducing the number of queue accesses. Simultaneously, it supports lightweight task stealing between schedulers to prevent low-priority tasks from starving indefinitely and improve global task balance.

[0139] The local wake-up and atomic control module divides idle coroutines into multiple local wake-up queues based on their ID hashes. Each queue is managed independently and wakes up in a targeted manner, significantly reducing global concurrency contention. When a task is enqueued, the wake-up operation of the corresponding queue is triggered according to priority. Signals are transmitted via a Compare-And-Swap (CAS) atomic operation, achieving lock-free, low-latency, and reliable wake-up. After being woken up, a coroutine atomically updates its own state and leaves the local queue, preventing multiple coroutines from competing for the same task.

[0140] The coroutine lifecycle management module pre-allocates coroutine stacks (stack size configurable between 8KB and 64KB) and control blocks based on a memory pool, avoiding frequent system calls and memory fragmentation. Coroutine states include initialization, running, idle, blocked, and pending destruction, with atomic operations ensuring state consistency. When a coroutine is destroyed, its stack memory and control block are reclaimed to the memory pool, and its association information with queues and cache pools is cleared to ensure no memory leaks.

[0141] The load monitoring module periodically calculates the length of the three-level task queue, the percentage of idle coroutines, and the local cache level. It compares the statistical results with scaling thresholds and cache level thresholds, and sends trigger signals to the decision-making module. Furthermore, this module records load change trends and supports dynamic threshold adjustments, thereby improving system stability under high load fluctuations.

[0142] Furthermore, the coroutine pool scheduling system of this application is divided into five layers, from bottom to top: resource management layer, basic scheduling layer, optimization and enhancement layer, task access layer, and monitoring and decision-making layer. Each layer has clear responsibilities and works collaboratively to achieve efficient operation of the coroutine pool.

[0143] 1. Resource Management Layer (Lowest Level) The system pre-allocates coroutine stack memory, control block memory, and queue node memory through a unified memory pool, supporting both fixed and variable-sized memory blocks, and possessing usage monitoring, defragmentation, and over-limit protection capabilities. The control block for each coroutine is uniformly managed by the memory pool, providing O(1) query and update interfaces. The global coroutine pool stores unallocated idle coroutines, serving as a resource buffer and fallback. The resource monitoring submodule monitors the memory pool usage, remaining control block quantity, and global pool level in real time, issuing an alarm to the upper layer when the usage exceeds 90%.

[0144] Each coroutine pool also maintains a doubly linked list of idle coroutines, with all idle coroutine control blocks chained together. Coroutines are retrieved from the head of the list during allocation and inserted at the tail during return. The total number of coroutines and the number of idle coroutines are maintained by atomic variables, supporting lock-free lookups. Furthermore, each coroutine pool records an "owner thread" (typically the scheduler or business thread that created the pool). When a coroutine is returned, if the current thread is not the owner, it is placed in the pool's remote return queue (protected by a lightweight lock). Before acquiring a coroutine, the owner thread will batch-transfer coroutines from the remote return queue to the idle list, enabling fast reuse.

[0145] 2. Basic Scheduling Layer The coroutine pool core engine is responsible for task queuing, coroutine scheduling, context switching, and basic execution. The system constructs a global lock-free queue (MPMC circular array) with high, medium, and low priorities, supporting concurrent enqueueing and dequeueing by multiple producers and consumers, all without locks or blocking. Each CPU core has an independent local cache pool that stores idle coroutines that can be immediately reused, enabling local reuse within the core. The scheduler retrieves coroutines according to the three resource priorities: first reading the local cache, then calling the task binding pool, and finally retrieving from the global pool. The basic scheduler implements user-mode context switching, saving only the program counter, stack pointer, and critical registers, with a single switch overhead of less than 50 nanoseconds. Idle coroutines are grouped and managed according to local wake-up queues for rapid wake-up.

[0146] To prevent short-term load fluctuations from causing frequent capacity expansion / contraction or water level write-back, all trigger actions are only executed after the threshold condition is met for three consecutive statistical periods. Data is collected once per period (e.g., 10 milliseconds), and historical data is stored in a circular buffer.

[0147] 3. Optimize the enhancement layer On top of basic scheduling, several enhancement mechanisms are provided: dynamic binding of tasks and coroutine pools to achieve isolated scheduling and Quality of Service (QoS) guarantee; adaptive water level write-back, which writes back excess coroutines to the global pool in batches when overloaded and replenishes them from the global pool when underloaded; partial wake-up and atomic control, group management and targeted wake-up, and use CAS to transmit signals; and CPU affinity scheduling, which binds coroutines to cores statically or dynamically to improve cache hit rate.

[0148] Blocking-aware compensation: When a coroutine performs blocking operations such as I / O, the scheduler changes its state from "running" to "blocked" and immediately retrieves another idle coroutine from the local cache or global pool to execute the task, preventing CPU idling. Blocked coroutines are put back into the local cache after completing their waiting period, awaiting the next scheduling.

[0149] 4. Task Access Layer (Top Layer) The system provides a unified task submission interface, supporting synchronous / asynchronous submission, task function encapsulation, parameter passing, and callback registration. User tasks are encapsulated into standard task structures, bound to a dedicated coroutine pool specified in the task information, and sent to the corresponding lock-free queue according to priority. It also supports task timeout settings, cancellation, status queries, and execution result returns.

[0150] 5. Monitoring and Decision-Making Level (Cross-Level) The global management center collects data in real time, including queue length, coroutine utilization, cache level, and scheduling latency, and automatically adjusts expansion and contraction thresholds, cache levels, and queue priorities. It records the number of coroutine creation / destruction attempts, task execution duration, scheduling attempts, and write-back attempts, and issues alerts when anomalies occur.

[0151] The scheduler retrieves tasks from the global lock-free queue in batches, with a configurable number of tasks retrieved at a time (typically 1-3). This reduces the number of CAS operations and avoids imbalances. Emergency expansion: When regular expansion still cannot meet the demand (zero global idle coroutines and a continuously growing task queue), the expansion step size is doubled (e.g., from 10 to 20) to quickly alleviate backlog.

[0152] Furthermore, this application provides another example to illustrate the embodiments of this application in more detail, as shown below: Taking a high-concurrency network request processing scenario as an example, the specific implementation process of this application is as follows (assuming the initial configuration: initial number of coroutines in the pool is 40, high load threshold is 40 (queue length), low load threshold is 80% (percentage of idle coroutines), expansion step size is 10, shrinkage ratio is 30%, minimum number of retained coroutines is 40, and all continuous period judgments use 3 periods): I. Coroutine Pool Initialization Phase: 1. Memory pre-allocation and free list construction: Upon system startup, based on the initial number of coroutines (40) and the stack size of each coroutine (16KB) specified in the configuration file, sufficient memory space is requested from the operating system in one go. This memory is divided into two parts: one part is used to store the stack space required for coroutine runtime, and the other part is used to store the control blocks of the coroutines (recording information such as the coroutine's ID, status, priority, and pool to which it belongs). The control blocks of all idle coroutines are linked together through a doubly linked list, with the head of the list stored in the coroutine pool structure. Two counters are maintained simultaneously: the total number of coroutines (40) and the current number of idle coroutines (40). Each coroutine control block records which coroutine pool it belongs to, so that the pool to which it belongs can be directly found when it is returned to its original location.

[0153] 2. Create a global coroutine pool and a scheduler for each CPU core: The system first creates a global coroutine pool object, which contains an idle list (for storing coroutines that have not yet been allocated to any local cache) and a remote return queue (used later). Based on the number of CPU cores (e.g., 4 cores), the same number of schedulers are created (each scheduler runs on a separate core). Each scheduler also has its own local cache pool, which is essentially a coroutine pool structure with its own idle list, high watermark (25), low watermark (8), and write-back threshold (4). During initialization, the 40 coroutines in the global pool are evenly distributed among the four local cache pools (10 in each), and their respective idle counts are updated.

[0154] 3. Establish a three-level lock-free task queue: Create three global lock-free queues (high, medium, and low priority), each implemented using a lock-free circular array with multiple producers and consumers. The queue length is configurable (e.g., 1024), supporting simultaneous enqueueing by multiple threads and dequeueing by multiple schedulers. CAS atomic operations are used throughout to avoid lock contention. Each queue node stores a pointer to a task structure.

[0155] 4. Initialize local wake-up groups: Idle coroutines are divided into multiple groups (e.g., 4 groups) by ID hash, and each group maintains a lock-free queue (multiple producers, single consumer). An atomic flag is also set for each group to quickly determine if any coroutines need to be woken up. This way, when a new task is enqueued, only certain groups can be woken up based on priority, avoiding the "thundering herd" effect caused by waking up all coroutines.

[0156] 5. Start the load monitoring thread: Create a separate monitoring thread with a statistical period of 10 milliseconds. Each period, it reads the length of all queues, the number of idle coroutines in all local cache pools, and the number of idle coroutines in the global pool, and records these data in a circular history buffer (retaining data from the most recent 10 periods). The monitoring thread is also responsible for comparing the current load with a preset threshold; when the conditions are met for multiple consecutive periods, it triggers expansion, contraction, or water level write-back operations.

[0157] 6. Create a dedicated coroutine pool for business applications: Dedicated coroutine pools are created for the payment, query, and logging services. Each dedicated pool is also an independent coroutine pool structure with its own idle list and owner thread (the owner thread is the service thread itself). These dedicated pools are added to the global pool list of their respective scheduling domains for unified management, but their resources are isolated from each other.

[0158] II. Task submission and binding pool allocation, such as Figure 3 As shown, Figure 3A schematic diagram of task submission and binding pool allocation provided for an embodiment of this application is shown below: 1. Task encapsulation and verification: When a user calls the task submission interface, the system allocates a task structure from the memory pool, fills it with a function pointer, parameters, priority (high / medium / low), and the bound dedicated coroutine pool ID (e.g., payment pool). Simultaneously, it verifies the validity of the task parameters (function pointer not null, stack size not exceeding configuration, etc.).

[0159] 2. Bind to a dedicated pool: In the task structure, set a coroutine pool binding field `co_pool`, pointing to the corresponding dedicated coroutine pool. This way, the scheduler knows which pool to retrieve coroutines from when executing the task. If the task does not specify a dedicated pool, the default global pool is used.

[0160] 3. Unlocked entry into the queue: Based on task priority, select the corresponding global lock-free queue (high, medium, low). Perform a lock-free enqueue operation: attempt to reserve a slot at the tail of the queue using a CAS atomic operation. If successful, write the task pointer to the slot and update the tail pointer; if the queue is full, return an error (which can be retried by the upper layer or the queue can be expanded). Upon successful enqueue, immediately trigger a wake-up operation.

[0161] 4. Local wake-up: The wake-up process determines which groups to wake up based on task priority: high-priority tasks wake up all groups, medium-priority tasks wake up half of the groups, and low-priority tasks wake up only one group. For each selected group, its `need_wake` flag is first set to 1 via atomic swap, and then the group's idle coroutine queue is checked for non-emptiness. If not empty, an idle coroutine is retrieved, its status is changed from "idle" to "runnable," and it is placed in the corresponding scheduler's local cache or run queue. The entire wake-up process uses CAS operations to ensure lock-free operation, and each coroutine is retrieved by only one waker to avoid duplication.

[0162] III. Load monitoring and dynamic expansion, such as Figure 4 As shown, Figure 4 A schematic diagram of a dynamic scaling process provided in this application embodiment is shown below: 1. Expansion trigger conditions: Each monitoring cycle counts the total task length across all global queues. If the total length exceeds the high load threshold (40), and the idle coroutine ratio is greater than 80% (indicating that although there are idle coroutines, the task backlog is faster), and this condition is met for three consecutive cycles, then a regular expansion is triggered. Additionally, if the number of idle coroutines in all local cache pools and the global pool is zero, and there is still a backlog of tasks in the queues, then an emergency expansion is immediately triggered.

[0163] 2. Specific procedures for routine capacity expansion: According to the expansion step size (10), allocate 10 new coroutine stacks and control blocks from the memory pool (if the memory pool is insufficient, request additional memory from the operating system first). Initialize the context of each new coroutine (set the entry function to the coroutine's main loop), and then allocate them to the local cache pools of each scheduler according to the CPU affinity strategy (e.g., polling), and add them to the corresponding free list. Update the total number of global coroutines and the free count of each local cache.

[0164] 3. Emergency expansion: During emergency expansion, the expansion step size is doubled (20 units), and a large block of memory is allocated directly from the operating system via memory mapping and added to the memory pool, then coroutines are created. After emergency expansion, the system will attempt to allocate coroutines to the waiting queue again.

[0165] IV. Three-level scheduling and local cache reuse: 1. Scheduler main loop: The scheduler thread on each CPU core continuously executes the following steps: First, it retrieves tasks in batches from the three-level priority queue (up to three at a time, trying them from highest to lowest priority); then, it retrieves an idle coroutine according to the three-level resource acquisition order; finally, it assigns the task to the coroutine for execution.

[0166] Order of acquiring Tier 3 resources: Level 1: Pop a goroutine from the free list of the current scheduler's local cache pool (using lock-free stack operations). If successful, return immediately.

[0167] Level 2: If the local cache is empty, check the dedicated coroutine pool bound to the current task (via the co_pool field in the task structure). If the dedicated pool exists and has an idle coroutine, retrieve one from its idle list (Note: The owner thread of the dedicated pool may be a business thread, but the scheduler thread needs to lock or use a remote return mechanism when accessing it. Here, a remote return queue and lock-free optimization for the owner thread are adopted).

[0168] Level 3: If the dedicated pool is also empty, then a coroutine is finally obtained from the free list of the global coroutine pool. After obtaining a coroutine, its status is immediately changed to "running", and its current scheduler is recorded.

[0169] 2. Coroutine execution of tasks: The scheduler switches the coroutine's context to the task function. After execution, the coroutine voluntarily relinquishes the CPU and returns to the scheduler. The scheduler checks the coroutine's status; if the task has completed normally, it changes the coroutine's status back to "idle" and prepares to return it to the scheduler.

[0170] 3. Batch task retrieval: To improve throughput, the scheduler retrieves 1 to 3 tasks from the queue at a time, stores them in a local mini-batch array, and then binds them to coroutines for execution one by one. This reduces the number of accesses to the global queue and lowers CAS contention.

[0171] V. Local cache overload and adaptive watermark write-back, such as Figure 5 As shown, Figure 5 A schematic diagram of an adaptive cache pool water level write-back process provided in this application embodiment is shown below: 1. Water level detection: Each local cache pool maintains an atomic variable for the current number of idle coroutines. The load monitoring module reads this value each cycle and compares it with the high watermark (25) and the low watermark (8). For smoothing, overload write-back is triggered only when the high watermark is exceeded for three consecutive cycles and the number of cycles exceeding the high watermark reaches the write-back threshold (4). Similarly, replenishment is triggered only when the high watermark is below the low watermark for three consecutive cycles.

[0172] 2. Specific operations for overload write-back: Calculate the number of goroutines to be written back: current free goroutine count minus the high watermark (e.g., 29 - 25 = 4). Pop four goroutines in batches from the local cache pool's free list (using lock-free batch pop operations), then add the control blocks of these goroutines one by one to the global goroutine pool's free list. If a dedicated goroutine pool is overloaded, write back to the global pool or its associated domain pool. The entire write-back process is atomic, does not interrupt running goroutines, and does not generate lock contention.

[0173] 3. Specific procedures for low-load replenishment: When the number of idle coroutines falls below the low watermark (e.g., 6), calculate the number that needs to be replenished: low watermark minus the current number (8 - 6 = 2). Take two idle coroutines from the global coroutine pool and add them directly to the idle list of the local cache pool. If the global pool is also insufficient, trigger expansion.

[0174] VI. Detailed Implementation of Remote Return and Lock-Free Wake-up: 1. Fast path and remote path for coroutine return: After a coroutine completes its task, it calls the return interface. First, it checks if the current thread is the owner thread of the pool to which the coroutine belongs (the owner is usually the scheduler thread or business thread that created the pool). If it is the owner thread, the coroutine control block is directly inserted into the head (or tail) of the pool's free list, and the free count is incremented. This is the fastest path, lock-free and contention-free. If it is not the owner thread (for example, the scheduler thread uses a coroutine from a dedicated pool, and the owner of the dedicated pool is a business thread), it cannot directly operate on the pool's free list, otherwise it will cause concurrent conflicts. In this case, a remote return queue is used: the coroutine control block is inserted into the pool's remote return queue (this queue is protected by a mutex lock, but return operations are infrequent, and the performance loss is acceptable). Before each time the owner thread retrieves a coroutine from its own pool, it first clears the remote return queue (i.e., moves all remotely returned coroutines to the free list), thus enabling remotely returned coroutines to be quickly reused.

[0175] 2. Group management with lock-free wake-up: Each wakeup group maintains a lock-free queue (containing the control block for idle coroutines) and an atomic flag. When a new task is enqueued, the group to be woken up is selected based on priority. For each selected group, the flag is atomically swapped from 0 to 1 (indicating a need for wakeup). If the swap is successful and the group's idle queue is not empty, a coroutine is popped and placed in the scheduler's local buffer. Due to the use of CAS, when multiple wakers operate on the same group simultaneously, only one can successfully pop a coroutine; the others will fail and retry or skip.

[0176] VII. Task completion and coroutine retrieval, such as Figure 6 As shown, Figure 6 A coroutine lifecycle state diagram provided for an embodiment of this application is shown below: 1. Coroutine return local caching: After the task is completed, the coroutine will return to the scheduler. The scheduler will change the coroutine's status from "running" to "idle" and then call the return interface to return the coroutine to the idle list (or remote return queue) of its pool.

[0177] 2. Scaling down triggers and gradual destruction: If the load monitoring module detects that all queues are empty and all coroutines are idle for three consecutive cycles, and the current total number of coroutines is greater than the minimum retention number (40), then it triggers a reduction in size. The reduction amount = (current total number - minimum retention number) × reduction ratio 30%. For example, if there are 50 coroutines and 10 more are needed, then 3 will be destroyed (or 10 can be destroyed according to the strategy, but the minimum retention number will be retained). Idle coroutines are collected from the global pool and each local cache pool (prioritizing the destruction of the least used ones), and then the destruction process is called: the coroutine is removed from the idle list, the stack memory and control block are released back to the memory pool, and the total coroutine counter is updated. After receiving the release, the memory pool attempts to merge adjacent idle memory blocks and optionally returns the entire memory block to the operating system.

[0178] 3. Resource monitoring and alerts: The memory pool internally maintains a utilization counter (allocated memory / total memory). When the utilization exceeds 90%, an alarm event is sent to the upper layer, which can then take measures such as rejecting new tasks, forcing expansion, or logging.

[0179] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0180] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above embodiments of the coroutine pool scheduling method.

[0181] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described embodiments of the coroutine pool scheduling method at runtime.

[0182] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0183] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described embodiments of the coroutine pool scheduling method.

[0184] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described coroutine pool scheduling method embodiments.

[0185] Any of the components, modules, units, parts, methods, and operations described herein can be implemented using software, firmware, hardware (e.g., fixed logic circuitry), manual processing, or any combination thereof. Alternatively or additionally, any functionality described herein can be executed at least in part by one or more hardware logic components, such as, but not limited to, a central processing unit (CPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), a system-on-a-chip (SoC), a complex programmable logic device (CPLD), a microprocessor (MCU), etc. The terms "system," "computing device," or "apparatus" as used herein encompass various means, devices, and machines for processing data, including, for example, one or more programmable processors, computers, SoCs, or combinations thereof. The apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or one or more combinations thereof. The aforementioned computer program (also known as a program, software, software application, app, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, object, or other unit suitable for a computing environment.

[0186] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0187] The foregoing has provided a detailed description of a coroutine pool scheduling method and electronic device. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A coroutine pool scheduling method, characterized in that, include: Construct a global coroutine pool, local cache pools for multiple processors, and a lock-free task queue; In response to the processing request of the target task, a dedicated coroutine pool associated with the target task is determined based on the task information of the target task. The local cache pool, the dedicated coroutine pool, and the global coroutine pool include coroutines for processing tasks. The coroutines in the local cache pool are allocated from the global coroutine pool. The target task is distributed to the corresponding lock-free task queue according to the preset task priority, and an idle coroutine is obtained in the order of the local cache pool, the dedicated coroutine pool, and the global coroutine pool, and the target task is executed based on the idle coroutine. Based on the number of tasks in the lock-free task queue and the number of idle coroutines in the local cache pool, the distribution of idle coroutines between the local cache pool and the global coroutine pool is adjusted, and / or the global coroutine pool is dynamically expanded or shrunk. The coroutine that has completed the target task will be returned to the local cache pool for reuse.

2. The coroutine pool scheduling method according to claim 1, characterized in that, The construction of the global coroutine pool, the local cache pools corresponding to multiple processors, and the lock-free task queue includes: Based on the preset number of coroutines and the preset coroutine stack size, coroutine stack space and control block space are allocated from the memory pool. The control blocks of idle coroutines are linked together through an idle linked list. The global coroutine pool includes the coroutine stack space and the control block space. The coroutine stack space is used to store coroutines, and the control block space is used to store control blocks. The control blocks are used to control the coroutines to execute tasks. Create schedulers corresponding to the plurality of processors, configure local cache pools for the plurality of schedulers respectively, and set a first threshold, a second threshold and a write-back threshold for each local cache pool; Distribute the idle coroutines in the global coroutine pool to multiple local cache pools; Create the lock-free task queue that supports concurrent enqueueing and dequeueing of multiple producers and consumers, and divide the idle coroutines in the local cache pool into multiple groups of local wake-up queues.

3. The coroutine pool scheduling method according to claim 2, characterized in that, The lock-free task queue includes a first priority queue, a second priority queue, and a third priority queue, wherein tasks in the first priority queue are executed first. The lock-free task queue uses a circular array and supports concurrent enqueueing of multiple producers and concurrent enqueueing and dequeueing of multiple consumers through atomic operations.

4. The coroutine pool scheduling method according to claim 1, characterized in that, Before determining the dedicated coroutine pool associated with the target task based on the target task's task information, the method further includes: Allocate a task structure for the target task from the memory pool, and populate the target task with the task function pointer, parameters, priority, and coroutine pool binding fields in the task structure; The coroutine pool pointed to by the coroutine pool binding field in the task structure is determined as the dedicated coroutine pool.

5. The coroutine pool scheduling method according to claim 2, characterized in that, The step of distributing the target task to the corresponding lock-free task queue according to the preset task priority, and obtaining idle coroutines in the order of the local cache pool, the dedicated coroutine pool, and the global coroutine pool includes: Idle coroutines are extracted from the idle list of the target local cache pool corresponding to the target scheduler, wherein the target scheduler is the scheduler that receives the processing request of the target task. In response to the absence of idle coroutines in the target local cache pool, an idle coroutine is obtained from the idle list of the dedicated coroutine pool based on the task information of the target task. In response to the absence of idle coroutines in the dedicated coroutine pool, an idle coroutine is obtained from the idle list of the global coroutine pool; Change the status of the acquired idle coroutine from idle to running, and record the scheduler to which the idle coroutine currently belongs.

6. The coroutine pool scheduling method according to claim 2, characterized in that, The step of adjusting the distribution of idle coroutines between the local cache pool and the global coroutine pool based on the number of tasks in the lock-free task queue and the number of idle coroutines in the local cache pool includes: The number of idle coroutines in the local cache pool is detected within a preset statistical period, and the preset statistical period in which the number of idle coroutines is greater than the first threshold is determined as the first statistical period. In response to the first statistical period reaching a first preset threshold number of consecutive times and the target number of coroutines reaching the write-back threshold, a first number of idle coroutines in the local cache pool are written back to the global coroutine pool. The target number of coroutines is the sum of the differences between the number of idle coroutines and the first threshold in consecutive first statistical periods. The preset statistical period in which the number of idle coroutines is less than the first threshold is determined as the second statistical period; In response to the second statistical period reaching the first preset threshold number of consecutive times, a second number of idle coroutines are extracted from the global coroutine pool and added to the local cache pool.

7. The coroutine pool scheduling method according to claim 6, characterized in that, The first quantity is the sum of the differences between the number of idle coroutines and the first threshold during the first consecutive statistical period; The second quantity is the sum of the differences between the first threshold and the number of idle coroutines in a continuous second statistical period.

8. The coroutine pool scheduling method according to claim 6, characterized in that, The dynamic scaling up and down process of the global coroutine pool includes: In response to the number of consecutive times in a preset statistical period where the number of tasks is greater than a preset task number threshold and the proportion of idle coroutines is less than a preset proportion threshold, a second preset number threshold is reached, and new idle coroutines are allocated from the memory pool to the global coroutine pool and / or the local cache pool according to the first expansion step size, wherein the proportion of idle coroutines in the local cache pool is the proportion of all coroutines. In response to the fact that the number of idle coroutines in the global coroutine pool and the local cache pool are both zero and the number of tasks is greater than a preset task number threshold, new idle coroutines are allocated from the memory pool to the global coroutine pool and / or the local cache pool according to a second expansion step size, wherein the second expansion step size is greater than the first expansion step size. In response to the number of consecutive statistical periods in which the number of tasks is less than the preset task number threshold and the proportion of idle coroutines is greater than the preset proportion threshold reaching a second preset number threshold, the idle coroutines in the global coroutine pool are destroyed according to a preset shrinkage ratio, and the minimum number of coroutines is retained.

9. The coroutine pool scheduling method according to claim 2, characterized in that, The step of returning the coroutine that has completed the target task to the local cache pool for reuse includes: In response to the fact that the coroutine that has completed the target task is a coroutine in its local cache pool, the coroutine that has completed the target task is written back to its local cache pool. If a coroutine that has completed the target task is not a coroutine in its local cache pool, the coroutine that has completed the target task is written to a remote return queue, and the coroutine that has completed the target task is written back to a dedicated coroutine pool or a global coroutine pool based on the remote return queue.

10. The coroutine pool scheduling method according to claim 5, characterized in that, The step of retrieving idle coroutines from the idle list of the target local cache pool corresponding to the target scheduler includes: Idle coroutines are divided into multiple local wake-up queues based on their identifier hashes, and atomic flags are set for each local wake-up queue. According to the preset task priority, a queue to be woken up is determined from the multiple groups of local wake-up queues, and the atomic flag position of the queue to be woken up is marked as to be woken up through atomic swap; In response to the presence of an idle coroutine in the wake-up queue, the idle coroutine is extracted from the wake-up queue, and its status is changed from idle to runnable.

11. The coroutine pool scheduling method according to claim 10, characterized in that, The step of determining the wake-up queue from the multiple groups of local wake-up queues according to the preset task priority includes: In response to the target task being in the first priority queue of the lock-free task queue, the first group of local wake-up queues in the multiple groups of local wake-up queues is determined as the wake-up queue; In response to the target task being in the second priority queue of the lock-free task queue, the second group of local wake-up queues in the multiple groups of local wake-up queues is determined as the wake-up queue; In response to the target task being in the third priority queue of the lock-free task queue, the third group of local wake-up queues in the multiple groups of local wake-up queues is determined as the wake-up queue; Among them, the number of the first group is greater than the number of the second group, which is greater than the number of the third group, and there is no idle coroutine in different wake-up queues.

12. The coroutine pool scheduling method according to claim 8, characterized in that, Before adjusting the distribution of idle coroutines between the local cache pool and the global coroutine pool based on the number of tasks in the lock-free task queue and the number of idle coroutines in the local cache pool, the method further includes: The number of tasks, the proportion of idle coroutines, the first threshold and the second threshold are collected at a preset statistical period to obtain the collection results, and the collection results are saved to the circular history buffer. The first threshold, the second threshold, the preset task quantity threshold, and the preset ratio threshold are dynamically adjusted based on the load change trend in the cyclic history buffer.

13. The coroutine pool scheduling method according to claim 2, characterized in that, The method further includes: Monitor the utilization rate of the memory pool; An alarm event is generated in response to the usage rate exceeding a preset usage rate threshold.

14. The coroutine pool scheduling method according to claim 2, characterized in that, The method further includes: In response to a blocking operation occurring during the execution of the target task by the coroutine, the coroutine's state is changed from running to blocked, and another idle coroutine is retrieved from the local cache pool or the global coroutine pool to execute the target task. After a blocked coroutine completes its waiting period, the blocked coroutine is put back into the local cache pool.

15. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the coroutine pool scheduling method as described in any one of claims 1 to 14 when executing the computer program.

Images