Task processing method, system, device, medium and product

Abstract

The embodiment of the invention provides a task processing method, system and device, a medium and a product, and the method comprises the steps: obtaining a token from a token library with a specified capacity in response to a task processing request; according to a target task processing request of the obtained token, calling callable threads pre-distributed in a thread pool, processing the target task processing request, and obtaining a task processing result, and a specified number of threads are pre-distributed in the thread pool. Through the cooperation of the token library and the thread pool, the conflict between the capacity guarantee and the resource utilization rate in the pulse flow scene can be effectively relieved, and the overall elasticity and stability of the system are remarkably improved.

Description

Technical Field

[0001] The embodiments in this specification relate to the technical field of task processing, and in particular to a task processing method, system, device, medium, and product. Background Technology

[0002] For the processing of high-concurrency tasks (e.g., concurrent access request processing), it is necessary to guarantee the number of tasks that the system can process concurrently in order to improve the system's capacity to handle concurrent tasks.

[0003] Traditional capacity assurance methods often rely on resource redundancy. While this redundancy design can handle extreme scenarios, resource utilization is generally low in non-extreme scenarios, leading to significant resource idleness and waste. There is a clear contradiction between ensuring system capacity and resource utilization: the more resources configured, the higher the peak traffic the system can handle, but the more severe the resulting resource idleness and waste. Summary of the Invention

[0004] In view of the above, embodiments of this specification provide a task processing method. One or more embodiments of this specification also relate to a task processing system, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a task processing method is provided, comprising: in response to a task processing request, obtaining a token from a token vault of a specified capacity; and for a target task processing request for which the token has been obtained, invoking a pre-allocated callable thread in a thread pool to process the target task processing request and obtain a task processing result, wherein a specified number of threads are pre-allocated in the thread pool.

[0006] According to a second aspect of the embodiments of this specification, a task processing system is provided, including a rate limiting layer and a service processing layer; the rate limiting layer is used to obtain a token from a token vault of a specified capacity in response to a task processing request; the service processing layer is used to call a pre-allocated callable thread in a thread pool to process the target task processing request and obtain a task processing result for the target task processing request for which the token has been obtained, wherein a specified number of threads are pre-allocated in the thread pool.

[0007] According to a third aspect of the embodiments of this specification, a computing device is provided, including: a memory and a processor; the memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, wherein the computer programs / instructions, when executed by the processor, implement the steps of the method of the first aspect described above.

[0008] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the method described in the first aspect above.

[0009] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the method described in the first aspect above.

[0010] In this embodiment, the token store's burst traffic allowance mechanism allows short-term burst traffic to quickly enter the system for processing directly through the token store. For persistent high traffic exceeding the system's safety threshold, it is effectively blocked due to token exhaustion. The thread pool pre-allocates and fixes the number of thread resources to ensure that requests can immediately enter the execution state and complete processing quickly when burst traffic arrives. Through the coordinated operation of the token store and thread pool, this embodiment effectively alleviates the contradiction between capacity assurance and resource utilization in burst traffic scenarios, significantly improving the overall resilience and stability of the system. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of an e-commerce checkout page provided in one embodiment of this specification;

[0012] Figure 2 This is a schematic diagram of pulse traffic on an e-commerce platform provided in one embodiment of this specification;

[0013] Figure 3 This is a flowchart illustrating a task processing method provided in one embodiment of this specification;

[0014] Figure 4 This is a schematic diagram of a task processing system provided in one embodiment of this specification;

[0015] Figure 5 This is a diagram illustrating a collaborative architecture between a rate limiting layer and a service processing layer, provided in one embodiment of this specification.

[0016] Figure 6 This is a flowchart of a service gateway rate limiting process provided in one embodiment of this specification;

[0017] Figure 7 This is a system rate limiting and processing architecture diagram provided in one embodiment of this specification;

[0018] Figure 8 This is a schematic diagram illustrating a QPS variation provided in one embodiment of this specification;

[0019] Figure 9 This is a schematic diagram of RT variation provided in one embodiment of this specification;

[0020] Figure 10 This is a schematic diagram illustrating another QPS variation provided in one embodiment of this specification;

[0021] Figure 11 This is another schematic diagram of RT variation provided in one embodiment of this specification;

[0022] Figure 12 This is a schematic diagram illustrating another QPS variation provided in one embodiment of this specification;

[0023] Figure 13 This is another schematic diagram of RT variation provided in one embodiment of this specification;

[0024] Figure 14 This is a schematic diagram illustrating another QPS variation provided in one embodiment of this specification;

[0025] Figure 15 This is another schematic diagram of RT variation provided in one embodiment of this specification;

[0026] Figure 16 This is a schematic diagram illustrating another QPS variation provided in one embodiment of this specification;

[0027] Figure 17 This is another schematic diagram of RT variation provided in one embodiment of this specification;

[0028] Figure 18 This is a schematic diagram illustrating another QPS variation provided in one embodiment of this specification;

[0029] Figure 19 This is a schematic diagram of the structure of a task processing device provided in one embodiment of this specification;

[0030] Figure 20 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0031] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0032] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0033] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0034] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0035] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0036] Pulse traffic refers to the phenomenon on e-commerce platforms where traffic surges rapidly from normal levels to several times the normal peak within a very short period, during scenarios such as order placement and coupon grabbing, before quickly returning to normal levels. The entire process typically lasts about 5 seconds and is characterized by its suddenness, high concurrency, and short duration.

[0037] Token bucket rate limiting algorithm: The system continuously adds tokens to the token bucket at a fixed rate (token generation rate). When the number of tokens in the bucket reaches the preset maximum capacity (bucket capacity), excess tokens are discarded and no longer accumulated. Whenever a new request arrives, the system attempts to retrieve a token from the token bucket. If there are still available tokens in the bucket, a token is retrieved and the request is allowed to continue processing; otherwise (if the bucket is exhausted), the request is rejected directly or rate limiting and degradation logic is triggered.

[0038] Core threads: The number of threads that are always alive in the thread pool, even when they are idle.

[0039] Non-core threads: When the number of tasks exceeds the core thread count, the thread pool will create additional threads (not exceeding the maximum thread count). These threads will be recycled after being idle for a period of time.

[0040] Maximum number of threads: The maximum number of threads that the thread pool is allowed to create.

[0041] Figure 1 A schematic diagram of an e-commerce checkout page according to one embodiment of this specification is shown. Figure 1 As shown, the checkout page 100 includes basic product information 110 and payment method 120. After clicking the Submit Order / Pay Now button 130, an order request will be generated.

[0042] Placing orders is the core function of e-commerce platforms, and the traffic models of different e-commerce platforms vary significantly. Traditional capacity assurance methods often rely on resource redundancy and rate limiting configurations. For example, the peak daily order volume of a platform can reach 30,000 orders per second, while during non-flash sales periods it drops to 3,000 orders per second, a difference of up to 10 times. Although this resource redundancy design can cope with extreme scenarios (such as flash sales or the "3, 2, 1" order fulfillment in live-streaming e-commerce), during non-flash sales periods, server resource utilization is generally low, resulting in a large amount of idle resources and significant resource waste.

[0043] For example, the platform's daily peak order volume is 100 orders per second. However, with rapid business growth, especially during promotional periods, livestreaming sales trigger surges in orders, with peak volumes soaring to 300 orders per second, and the upward trend is significant. It is projected that during another promotional period, peak order volumes will exceed 1000 orders per second. Faced with this highly volatile traffic model, traditional resource redundancy and rate limiting strategies would require reserving several times the server resources to handle peak traffic, resulting in significant resource waste during off-peak hours and leading to low cost efficiency.

[0044] Figure 2 This diagram illustrates a pulse traffic flow of an e-commerce platform according to an embodiment of this specification. The traffic volume refers to the total number of orders placed within a single second. Figure 2 This includes three types of statistics: the total number of transactions per second, the number of orders placed, and the number of live-stream sales. The total number equals the sum of the number of orders placed and the number of live-stream sales. For example... Figure 2 As shown, on June 15th, the traffic volume did not exceed 100 for most of the time, with only one data point (lasting approximately 5 seconds) approaching 300. If resource allocation is based on a capacity of 300 transactions per second, only this one data point allows for full resource utilization. Despite the increasing frequency of order pulses, this traditional capacity allocation strategy still suffers from significant resource waste when dealing with pulsed traffic.

[0045] There is a clear contradiction between ensuring system capacity and resource utilization: the more resources configured, the higher the peak queuings per second (QPS) the system can handle, but the more severe the resource idleness and waste. If resources are simply planned according to peak traffic, then in pulse traffic scenarios (such as e-commerce order placement, coupon grabbing, etc.), most resources will be severely wasted during off-peak periods. Therefore, it is necessary to find an efficient balance between capacity assurance and resource utilization to alleviate this contradiction to the greatest extent possible.

[0046] The embodiments in this specification demonstrate that by constructing a system with safe water level handling capabilities, it is possible to effectively support short-term pulse flow surges of several times the normal flow rate without increasing additional resource investment.

[0047] This specification provides a task processing method, and also relates to a task processing system, a request processing method, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0048] See Figure 3 , Figure 3 A flowchart of a task processing method according to an embodiment of this specification is shown, which specifically includes the following steps.

[0049] Step 302: In response to the task processing request, obtain a token from a token vault of the specified capacity.

[0050] In this context, a task processing request refers to a request that requires the processing of a task, which can refer to any type of task that requires data processing. For example, in an e-commerce platform, a task processing request could be an order placement request or a coupon grabbing request; in a social media platform, a task processing request could be a request to access trending news. The traffic from task processing requests is characterized by high bursts, extremely high concurrency, and very short duration, typically lasting only a few seconds.

[0051] A task processing request can be a single request or multiple concurrent task processing requests. The token vault stores tokens, and its storage capacity is fixed.

[0052] Among them, the token bucket rate limiting model is used to finely control the rate at which traffic enters the system. By utilizing the token bucket's burst traffic allowance mechanism, short-term pulse traffic can be directly and quickly entered into the system for processing through the token bucket. For high traffic that continuously exceeds the system's safety level, it will be effectively blocked or delayed as the tokens are exhausted, thereby preventing the service from avalanche due to continuous over-threshold requests.

[0053] When a task processing request arrives, a token needs to be obtained from the token store for the request. Only tasks with a token can proceed with further processing. The token store stores the tokens required for the request, and tokens are continuously added to the token store based on preset rules. The token store has a fixed capacity. For example, the token store can be a token bucket.

[0054] In some embodiments, a task processing request that arrives at a certain time is a single request, and in response to the request, a token is obtained for the request from a token vault of a specified capacity.

[0055] In some embodiments, multiple task processing requests arrive at a time. In response to the multiple task processing requests, a token needs to be obtained from the token store for each task processing request, wherein a single task processing request corresponds to a single token.

[0056] In some embodiments, the capacity of the token store can be set directly according to business needs.

[0057] In some embodiments, the token store capacity is determined by considering the token generation rate and caching time. For example, if 475 tokens are generated per second and the allowed caching window is 5 seconds, the token store can hold a maximum of 475 * 5 = 2375 tokens. For order requests, the maximum number of orders processed per second is 2375. The number of tokens generated per second and the token store capacity are set according to different business scenarios and resource characteristics. The token store capacity should be set to ensure normal access for short-duration burst traffic.

[0058] In some embodiments, it is optimal for the token library to accumulate tokens generated over a maximum of 5 seconds. Since the server is developed using Java, when a short-duration traffic pulse of 5 seconds occurs, it will only trigger a small amount of Young Generation Garbage Collection (Young GC) and will not trigger Old Generation Garbage Collection (Old GC) or a large number of Young GCs, thereby avoiding service crashes or sharp performance degradation due to a sudden increase in GC pressure.

[0059] In some embodiments, tokens are generated at a uniform rate and stored in a token vault. For example, a token generation rate of 475 tokens / second means that 475 tokens are generated per second, and that is, a maximum of 475 tokens can be stored in the token vault per second.

[0060] In some embodiments, the token generation rate is consistent with the token vault capacity. For example, the token vault capacity is 2375, and the token generation rate is 2375 tokens / second, meaning that the token vault can store 2375 tokens every second.

[0061] The system continuously adds tokens to the token pool at a fixed rate (token generation rate). When the number of tokens in the token pool reaches a preset capacity, excess tokens are discarded and no longer accumulate.

[0062] Step 304: For the target task processing request that has obtained the token, call the callable thread pre-allocated in the thread pool to process the target task processing request and obtain the task processing result, wherein the thread pool pre-allocated a specified number of threads.

[0063] The target task processing request is the task processing request that has obtained the token, and the callable thread is the thread that is currently idle and can be assigned the target task processing request. The task processing result includes whether the task processing was successful or failed.

[0064] The system employs a thread pool mechanism for request scheduling and concurrency control. The thread pool pre-allocates and fixes the number of threads to ensure that requests can immediately enter the execution state and be processed quickly when bursts of traffic arrive. To avoid request backlog, a queuing mechanism is not used; the thread pool's task queue length is zero. Once the number of concurrent requests exceeds the preset thread pool limit, the excess requests will immediately trigger a rejection policy. Furthermore, all worker threads are created at once during thread pool initialization, effectively avoiding the additional overhead caused by dynamically creating threads when requests arrive, thereby further reducing system response latency. This design ensures that the service can handle a large number of requests promptly during short bursts of traffic, avoiding response latency and resource exhaustion caused by queue waiting. Combined with the aforementioned token bucket rate limiting model, it effectively avoids service resource exhaustion under continuous high load scenarios, achieving more accurate and stable service quality assurance.

[0065] In some embodiments, if the number of concurrent task processing requests exceeds the number of tokens stored in the token store, only a portion of the task processing requests can obtain tokens for subsequent processing. For example, if the number of concurrent task processing requests is 500, and the token store contains 475 tokens, then only 475 task processing requests can obtain tokens. This can be achieved by randomly selecting 475 requests from the 500 requests, or by selecting the 475 requests that were requested earlier in the order they occurred.

[0066] In some embodiments, if the number of concurrent task processing requests at any given time does not exceed the number of tokens stored in the token store, then all concurrent task processing requests can obtain a token for subsequent processing. For example, if the number of concurrent task processing requests is 500 and the number of tokens in the token store is 575, then all 500 task processing requests can obtain a token.

[0067] In some embodiments, for a target task processing request that has obtained a token, a pre-allocated callable thread in the thread pool is invoked to process the target task processing request and obtain a task processing result. The thread pool pre-allocates a specified number of threads. For example, if there are 475 target task processing requests that have obtained tokens, and 500 threads are pre-allocated in the thread pool, with 480 threads in an idle state, then 475 callable threads can be selected from the idle threads to process the 475 target task processing requests and obtain a task processing result. There is a one-to-one relationship between threads and target task processing requests; that is, one thread can only process one target task processing request at a time, and one target task processing request can only be processed by one thread at a time. Threads in an idle state that can process tasks are callable threads. The task processing result may be a message indicating successful task processing.

[0068] In some embodiments, the number of callable threads is less than the number of target task processing requests. In this case, the callable threads in the thread pool can only process a portion of the target task processing requests to obtain task processing results. The remaining target task processing requests are rejected based on a rejection policy. For example, if 475 target task processing requests have obtained tokens, and 500 threads are pre-allocated in the thread pool with 300 threads in an idle state, then the 300 idle callable threads are invoked to process these 300 target task processing requests and obtain task processing results. The remaining 175 target task processing requests are rejected. This can be achieved by randomly selecting 300 requests from the 475 target task processing requests, or by processing the 300 requests that appeared earlier in the request time sequence.

[0069] In this embodiment, the token store's burst traffic allowance mechanism allows short-term burst traffic to quickly enter the system for processing directly through the token store. For persistent high traffic exceeding the system's safety threshold, it is effectively blocked due to token exhaustion. The thread pool pre-allocates and fixes the number of thread resources to ensure that requests can immediately enter the execution state and complete processing quickly when burst traffic arrives. Through the coordinated operation of the token store and thread pool, this embodiment effectively alleviates the contradiction between capacity assurance and resource utilization in burst traffic scenarios, significantly improving the overall resilience and stability of the system.

[0070] In some embodiments, before retrieving a token from a token vault of a specified capacity in response to a task processing request, the method further includes: generating a token at a preset rate and storing the generated token in a token vault.

[0071] Not only is the capacity of the token vault limited, but the rate at which tokens are added to the vault is also limited. The preset rate refers to the pre-defined token generation rate. The system continuously adds tokens to the token vault at this preset rate. When the number of tokens in the vault reaches the preset capacity, excess tokens are discarded and no longer accumulated. The token generation process continues uninterrupted.

[0072] In some embodiments, tokens are generated and stored in a token vault at a preset rate. For example, the preset rate is 475 tokens / second, meaning that 475 tokens are generated and stored in the token vault per second. It is understood that if more than 475 tokens are consumed in the token vault within a certain second, only 475 tokens can be stored in the token vault in the next second.

[0073] The number of tokens generated per second is set according to different business scenarios and resource characteristics. Typically, the preset rate should satisfy normal access from non-pulsating traffic (normal traffic).

[0074] For example, the token vault has a capacity of 2375 and a preset rate of 475, generating 475 tokens per second and storing them in the token vault. When the number of tokens in the token vault reaches the preset maximum capacity, the excess tokens will be discarded and will no longer accumulate.

[0075] In the embodiments described in this specification, tokens are generated and stored in the token library at a preset rate. The token generation rate is limited, which can effectively prevent the number of tokens in the token library from remaining at a high level, resulting in excessive traffic entering the system and improving the stability of the system.

[0076] In some embodiments, before retrieving tokens from a token vault of a specified capacity in response to a task processing request, the method further includes: creating a specified number of threads and constructing a thread pool based on the threads.

[0077] The thread pool pre-allocates a specified number of threads, which are created in advance. After creating the specified number of threads, the thread pool can be built based on these pre-created threads.

[0078] In some embodiments, these specified number of threads are all created at once during thread pool initialization, which can effectively avoid the additional overhead caused by dynamically creating threads when a request arrives, thereby further reducing system response latency.

[0079] In some embodiments, these specified number of threads are created in advance before the first request arrives.

[0080] In some embodiments, the core thread count is set to equal the maximum thread count for the thread pool configuration, with the specific value determined based on actual business needs. For example, the default maximum thread count is set to 500.

[0081] After all threads are created during initialization, since the core thread count equals the maximum thread count, all threads will remain alive in subsequent stages. Even if a thread is idle, it will not be reclaimed unless the parameter allowCoreThreadTimeOut(true) is set. Setting the value of allowCoreThreadTimeOut to true indicates that core threads are allowed to be reclaimed after being idle for a preset time.

[0082] In the embodiments described in this specification, a specified number of threads are created, and a thread pool is built based on these threads. This effectively avoids the additional overhead caused by dynamically creating threads when a request arrives, thereby further reducing system response latency.

[0083] In some embodiments, the target task processing request includes multiple concurrent target task processing requests; after obtaining a token from a token vault of a specified capacity in response to the task processing request, the method further includes: for the multiple target task processing requests for which the token has been obtained, if the number of requests for the multiple target task processing requests is greater than the number of threads that can be called, determining from the multiple target task processing requests the task processing requests to be rejected, and rejecting the task processing requests to be rejected.

[0084] It should be noted that pending rejection requests are target task processing requests that have not been assigned a callable thread among multiple target task processing requests.

[0085] At any given moment, there may be more than one request, including multiple concurrent requests for processing target tasks.

[0086] This specification explicitly states that no queuing mechanism is used in the embodiments; that is, the task queue length of the thread pool is 0. Once the number of concurrent requests exceeds the preset thread pool limit, the excess requests will immediately trigger a rejection policy. `queueSize=0` indicates the use of a zero-capacity synchronous queue; tasks will not enter the queue to wait but will be directly submitted to the thread for execution. If there are no idle threads, new tasks will trigger a rejection policy.

[0087] Specifically, for multiple target task processing requests that have obtained tokens, it is necessary to determine whether all of these target task processing requests can be processed by a thread. If the number of multiple target task processing requests is less than or equal to the number of callable threads, all of these target task processing requests can be processed; if the number of multiple target task processing requests is greater than the number of callable threads (only some target task processing requests can be processed), task processing requests to be rejected are identified from the multiple target task processing requests, and these rejected task processing requests are rejected.

[0088] For example, if the current thread pool has 300 callable threads, and the number of newly acquired token-granting target task processing requests is 400, then only 300 of these 400 requests can be processed. Therefore, 100 requests are selected from these 400 and rejected. These 100 requests can be determined based on the order in which they arrive or randomly.

[0089] The rejection policy is AbortPolicy, a strict rejection policy suitable for scenarios where explicit handling of task submission failures is required. When all threads in the thread pool are busy and unable to accept new tasks, a RejectedExecutionException will be thrown directly.

[0090] Specifically, according to the AbortPolicy, for the portion of the target task processing requests that exceed the limit, a direct rejection exception is thrown. The direct rejection exception is a RejectedExecutionException.

[0091] For example, when a task processing request is submitted, if the number of currently running threads is less than the number of threads in the thread pool, a callable thread in the thread pool will be invoked to execute the task. If the number of currently running threads has reached the number of threads allocated by the thread pool, the task processing request will be directly rejected, and a RejectedExecutionException exception will be thrown.

[0092] In the embodiments of this specification, when the number of requests for multiple target task processing requests exceeds the number of callable threads, task processing requests to be rejected are determined from the multiple target task processing requests, and these requests are rejected. In this way, when the thread pool is fully loaded, excess requests can be directly rejected, preventing requests from queuing in the thread queue for a long time and avoiding the risk of uncontrollable service response time (RT) caused by thread accumulation. When the system reaches its bottleneck, it can provide rapid feedback instead of waiting indefinitely, reducing the possibility of backend resource exhaustion caused by excessively long queues, thereby ensuring that the system can maintain stable and predictable response performance in high-concurrency scenarios.

[0093] In some embodiments, after obtaining a token from a token vault of a specified capacity in response to a task processing request, the method further includes: rejecting the initial task processing request if no token is obtained.

[0094] When a task processing request arrives, a token needs to be obtained for each task processing request. Only the target task processing request that has obtained a token can be processed. Therefore, the initial task processing request that has not obtained a token will be rejected.

[0095] In some embodiments, a rate-limiting return status code and a rate-limiting return content are set, and the rate-limiting status code and rate-limiting content are output when the initial task processing request is rejected.

[0096] For example, the token store capacity is 2375, and the current token store contains 2375 tokens. At this time, there are 2500 concurrent task processing requests, so only 2375 requests can obtain tokens. The remaining 125 requests are directly rejected.

[0097] In the embodiments described in this specification, initial task processing requests that have not obtained a token are rejected. In this way, short-term pulse traffic can be quickly processed by directly entering the system through the token store, while high traffic that continuously exceeds the system's safety threshold will be effectively blocked as tokens are exhausted, thereby preventing service avalanche due to continuous over-threshold requests.

[0098] In some embodiments, after obtaining a token from a token vault of a specified capacity in response to a task processing request, the method further includes: for an initial task processing request for which no token has been obtained, allocating a token for the initial task processing request from the tokens to be stored, and adding the remaining tokens to be stored to the token vault.

[0099] Among them, the tokens to be stored are tokens generated by the token generation unit but not added to the token library, and the remaining tokens to be stored are tokens to be stored that have not been assigned to the initial task processing request.

[0100] The tokens in the token vault are generated and added to the token vault by the token generation unit. After the token generation unit generates tokens, it does not immediately add all the generated tokens to the token vault. Tokens that are not added to the token vault are temporarily stored in the token generation unit as tokens to be stored.

[0101] The tokens to be stored in the token generation unit can be generated according to a preset token generation rate or according to other rules; this specification does not limit this. The tokens to be stored can be remaining tokens previously added to the token library, or newly generated tokens that have not yet been added to the token library.

[0102] For the initial task processing request that has obtained the token to be stored, subsequent processing can continue.

[0103] In some embodiments, the tokens to be stored generated by the token generation unit are temporarily stored within the token generation unit. Only when an initial task processing request arrives without obtaining a token will a token be allocated to the initial task processing request from the tokens to be stored, and the remaining tokens to be stored will be added to the token library. Tokens to be stored that ultimately fail to be added to the token library can be retained in the token generation unit indefinitely, or a token lifespan can be set, after which the token to be stored will be deleted from the token generation unit, or it can be directly discarded. For example, if there are 500 tokens to be stored in the token generation unit, and 300 initial task processing requests have not obtained tokens, then 300 tokens can be directly allocated to the initial task processing requests from the 500 tokens to be stored, and the remaining 200 tokens to be stored can be added to the token library.

[0104] In some embodiments, if the number of initial task processing requests that have not acquired a token is greater than the number of tokens to be stored, then initial task processing requests that have not acquired a token are identified as having been allocated tokens to be stored; the remaining initial task processing requests that have not been allocated tokens are directly rejected. In this case, the number of remaining tokens to be stored is 0.

[0105] In the embodiments described in this specification, for initial task processing requests that have not obtained a token, a token is allocated to the initial task processing request from the tokens to be stored, and the remaining tokens to be stored are added to the token pool. In this way, by allocating a token to the initial task processing request from the tokens to be stored, the probability of the request obtaining a token is increased, thereby increasing the service response rate. By adding the remaining tokens to be stored to the token pool, the number of tokens in the token pool can be effectively increased.

[0106] In some embodiments, before allocating a token for the initial task processing request from the tokens to be stored and adding the remaining tokens to the token library, the method further includes: obtaining the access status of the token generation unit; allocating a token for the initial task processing request from the tokens to be stored and adding the remaining tokens to the token library, including: when the access status is an allowed status, allocating a token for the initial task processing request from the tokens to be stored and adding the remaining tokens to the token library.

[0107] To avoid frequent access to the token generation unit for initial task processing requests that have not obtained a token, the system sets an access status for the token generation unit, including an allowed status and a prohibited status. The access status indicates whether the system can access the token generation unit. When the access status is allowed, a token is allocated to the initial task processing request from the tokens to be stored, and the remaining tokens to be stored are added to the token store.

[0108] In some embodiments, when the access status is prohibited, the initial task processing request is refused (a token is not allocated for the initial task processing request from the tokens to be stored, and the remaining tokens to be stored are added to the token vault).

[0109] In some embodiments, the access period of the token generation unit is set (e.g., the period is 1 second), and the access status is set to allowed for a portion of the time within each access period (e.g., within 100 milliseconds), and the access status is set to prohibited for the rest of the access period.

[0110] In some embodiments, the access status is set to allowed at the beginning of each access cycle, waiting for the arrival of an initial task processing request that has not obtained a token. After allocating a token to the initial task processing request from the tokens to be stored and adding the remaining tokens to be stored to the token library, the access status is set to prohibited until the start of the next access cycle, at which point the access status is set to allowed again.

[0111] In some embodiments, the token generation unit can be a Redis database, which is a storage system that supports multiple data structures.

[0112] In some embodiments, an access switch is provided, and the access state of the token generation unit is controlled by controlling the state of the access switch. For example, when the access switch is open, the access state of the token generation unit is prohibited; when the access switch is closed, the access state of the token generation unit is permitted.

[0113] In some embodiments, when the access switch is in the allowed state and the number of tokens to be stored is less than the number of initial task processing requests that have not obtained tokens, a portion of the initial task processing requests are identified from the initial task processing requests and tokens are allocated; for the remaining initial task processing requests, the initial task processing requests are rejected.

[0114] In the embodiments described in this specification, the access status of the token generation unit is obtained. If the access status is allowed, a token is allocated to the initial task processing request from the tokens to be stored, and the remaining tokens to be stored are added to the token library. In this way, by controlling the access status of the token generation unit, access control and token addition for the initial task processing request can be performed, avoiding frequent access to the token generation unit.

[0115] In some embodiments, before allocating a token for the initial task processing request from the tokens to be stored and adding the remaining tokens to the token library, the method further includes: generating tokens to be stored at a preset rate if the current timestamp and the preset time point meet the token generation conditions.

[0116] Here, the current timestamp represents the current moment, and the preset time point is the moment the token is generated. For example, the token generation period is set to 1 second, divided into 1000 moments. Tokens are generated at the initial moment of each second according to a preset rate, and no tokens are generated at other moments within the second. The token generation condition is used to determine whether a token needs to be generated at the current timestamp.

[0117] In some embodiments, if the current timestamp matches the preset time point, it is determined that the token generation conditions are met, and the token generation unit automatically generates a token to be stored. If the current timestamp does not match the preset time point, the token generation unit will not generate a token to be stored.

[0118] In some embodiments, each token generation cycle is divided into multiple preset time points, and tokens are generated at each preset time point within each preset cycle according to a preset rate. For example, if 10 preset time points are divided within a 1-second cycle, and 500 tokens need to be generated to be stored within the 1-second cycle, 50 tokens can be generated at each preset time point. The preset time points within the cycle can be evenly distributed or set according to requirements.

[0119] In some embodiments, the token generated at a preset time point and the token stored before the token generation unit together constitute the token to be stored.

[0120] In the embodiments described in this specification, when the current timestamp and the preset time point meet the token generation conditions, a token to be stored is generated according to a preset rate. This allows for fine-grained control over token generation.

[0121] In some embodiments, after allocating a token for the initial task processing request from the tokens to be stored and adding the remaining tokens to the token library, the method further includes updating the access status of a preset time point and the token generation unit.

[0122] The token generation unit generates tokens to be stored at preset time points in each cycle. After generating tokens to be stored in each cycle, the preset time points need to be updated to obtain the preset time points for the next cycle, so that tokens to be stored can be generated at the preset time points in each cycle.

[0123] The access status of the token generation unit varies at different times. In each cycle, the access status of the token generation unit needs to be updated to ensure that the access status remains normal within each cycle.

[0124] In some embodiments, the token generation cycle is 1 second, the preset time point is the initial moment of each second, and the access state of the token generation unit is set to be in an allowed state at the beginning of each cycle, waiting for access from initial task processing requests that have not yet acquired a task. After allocating a token for the initial task processing request from the tokens to be stored and adding the remaining tokens to the token library, the access state is set to a prohibited state until the next cycle arrives, at which point the access state is set to an allowed state again. For example, after allocating a token for the initial task processing request from the tokens to be stored and adding the remaining tokens to the token library, the preset time point and the access state of the token generation unit are updated, the preset time point is updated to the preset time point of the next cycle, and the access state is adjusted to a prohibited state within the current cycle.

[0125] In the embodiments described in this specification, by updating the preset time points and the access status of the token generation unit in a timely manner, it can be ensured that the token generation unit accurately generates tokens in each cycle and that precise control over the access status of the token access unit can be achieved.

[0126] See Figure 4 , Figure 4 A schematic diagram of a task processing system 400 according to an embodiment of this specification is shown. The task processing system 400 includes a rate limiting layer 410 and a service processing layer 420.

[0127] Rate limiting layer 410 is used to obtain tokens from a token vault of a specified capacity in response to task processing requests;

[0128] The service processing layer 420 is used to process the target task processing request for which a token has been obtained by calling the callable threads pre-allocated in the thread pool to process the target task processing request and obtain the task processing result. The thread pool pre-allocated a specified number of threads.

[0129] Among them, by using the two dimensions of the rate limiting layer and the service processing layer, it is possible to effectively support short-term pulse traffic surges of several times without increasing additional resource investment.

[0130] The rate limiting layer employs a token library rate limiting model to finely control the rate at which traffic enters the system. By utilizing the token library's burst traffic allowance mechanism, short-term pulse traffic can be directly and quickly entered into the system for processing. For high traffic that continuously exceeds the system's safety threshold, it will be effectively intercepted or delayed as the tokens are exhausted, thereby preventing the service from avalanche due to continuous over-threshold requests.

[0131] The service processing layer employs a thread pool mechanism for request scheduling and concurrency control. The thread pool pre-allocates and fixes the number of threads to ensure that requests can immediately enter the execution state and be processed quickly when burst traffic arrives. To avoid request backlog, a queuing mechanism is explicitly not used; the thread pool's task queue length is zero. Once the number of concurrent requests exceeds the preset thread pool limit, the excess requests will immediately trigger a rejection policy. Furthermore, all worker threads are created at once during thread pool initialization, effectively avoiding the additional overhead caused by dynamically creating threads when requests arrive, thereby further reducing system response latency.

[0132] In the embodiments described in this specification, the coordinated operation of the rate limiting layer and the service processing layer can effectively alleviate the contradiction between capacity guarantee and resource utilization in pulse traffic scenarios, and significantly improve the overall elasticity and stability of the system.

[0133] See Figure 5 , Figure 5 This diagram illustrates a collaborative architecture between a rate limiting layer and a service processing layer according to one embodiment of this specification. Figure 5As shown, in the rate limiting layer 510, tokens are generated at a uniform rate and stored in a token bucket, which has a maximum capacity of tokens. Upon receiving a request, a token is retrieved from the token bucket. Requests that successfully obtain a token are forwarded to the service processing layer for processing. Requests that fail to obtain a token are rejected, and a response is returned. In the service processing layer 520, a thread pool pre-allocates a fixed number of threads. Incoming requests are directly assigned to pre-created threads, avoiding increased time consumption due to thread creation and task queuing. Once the task is completed, an error is returned if no idle threads are available.

[0134] In some embodiments, the task processing system further includes a business gateway and a service component; a rate limiting layer is deployed on the business gateway and the service component respectively; and a service processing layer is deployed on the service component.

[0135] In the production system, the rate limiting layer is deployed at both the business gateway and the service component, while the service processing layer is deployed only at the service component. The business gateway primarily handles external Application Programming Interface (API) requests, while the service component primarily handles internal service calls. For example, the service component could be a Remote Procedure Call (RPC) service.

[0136] In this system, rate limiting layers are deployed in both the business gateway and service components to limit task processing requests twice. Without additional expansion of hardware / computing resources, this system can smoothly absorb short-term traffic surges (pulses) and avoid sudden increases in backend service pressure caused by instantaneous request peaks, which could lead to system jitter or unavailability, thereby improving overall throughput and availability.

[0137] In the embodiments described in this specification, token library rate limiting is uniformly adopted at the entry points of business gateways and service components. Without additional expansion of hardware / computing resources, it can smoothly absorb short-term traffic surges (pulses), avoid instantaneous request peaks causing a sudden increase in backend service pressure, and prevent system jitter or unavailability, thereby improving overall throughput and availability.

[0138] In some embodiments, the task processing system further includes a routing layer, a rate limiting layer including multiple rate limiting units with different deployment information, and a service processing layer including multiple service processing units with different deployment information; the routing layer is used to filter out the target rate limiting unit and the target service processing unit from the multiple rate limiting units and the multiple service processing units respectively according to the user identifier corresponding to the task processing request; the target rate limiting unit is used to obtain a token from a token pool of a specified capacity in response to the task processing request; the target service processing unit is used to call a pre-allocated callable thread in the thread pool to process the target task processing request and obtain the task processing result for the target task processing request that has obtained the token, wherein a specified number of threads are pre-allocated in the thread pool.

[0139] The task processing system adopts a modular architecture. The rate limiting layer includes multiple rate limiting units with different deployment information, and the service processing layer includes multiple service processing units with different deployment information. Different rate limiting units are deployed in different modular architectures, and different service processing units are deployed in different modular architectures.

[0140] Among these, multiple rate limiting units with different deployment information can refer to different geographical locations of the rate limiting units, or different rate limiting methods (including rate limiting size) of the rate limiting units. Multiple service processing units with different deployment information can refer to different geographical locations of the service units, or different rate limiting methods or request processing methods of the service units.

[0141] The task processing system also includes a routing layer. Based on the user identifier corresponding to the task processing request, the routing layer selects the target rate limiting unit and the target service processing unit from multiple rate limiting units and multiple service processing units. The routing layer calculates a hash result based on the user identifier; different hash results correspond to different target rate limiting units and target service processing units.

[0142] For example, the rate limiting layer includes rate limiting unit A and rate limiting unit B, and the service processing layer includes service processing unit A and service processing unit B. Rate limiting unit A and service processing unit B are in the same unitized architecture. After a task processing request arrives, the routing layer calculates a hash result based on the user identifier (user ID) corresponding to the task processing request, and selects the target rate limiting unit (e.g., rate limiting unit A) and the target service processing unit (e.g., service processing unit A) from rate limiting unit A and rate limiting unit B based on the hash calculation result. Then, in response to the task processing request, rate limiting unit A obtains a token from a token store of a specified capacity. For the target task processing request that has obtained the token, service processing unit A calls a pre-allocated callable thread in the thread pool to process the target task processing request and obtain the task processing result.

[0143] In the embodiments of this specification, by setting a routing layer, different task processing requests can be routed to different unitized architectures, and then the corresponding target rate limiting unit and target service processing unit can be called to process the task processing requests, thereby improving the adaptability of the unitized architecture of the task processing system.

[0144] Figure 6 This document illustrates a flowchart of a service gateway rate limiting process provided in an embodiment of this specification. In this embodiment, the service gateway only involves rate limiting functionality. Figure 6 As shown, the gateway edith pod includes a memory token bucket for a certain API and a switch for accessing Redis. The specific process is as follows:

[0145] Step 602: Access the API's memory token bucket.

[0146] When traffic enters the system, it first accesses the memory token bucket. For API interfaces with configured rate limiting policies, a unique token bucket instance is allocated for it in local memory.

[0147] Step 604: Access downstream services.

[0148] Step 606: Check the Redis access switch for the API.

[0149] If a token is obtained, proceed to step 604; if a token is not obtained, proceed to step 606.

[0150] Step 608: Check if it is necessary to regenerate the token.

[0151] After step 606, if the switch is off, rate limiting is performed directly and the corresponding result is returned; if the switch is on, step 608 is executed to continue accessing Redis. When accessing Redis, Lua script logic is executed to determine whether a token needs to be regenerated based on the current timestamp and a preset time point. If the current timestamp matches the preset time point, a token regeneration is required; otherwise, it is not required.

[0152] Step 610: Generate a new token.

[0153] Step 612: Consume the token.

[0154] After step 608 is executed, if it is necessary to regenerate the token, step 610 is executed to generate the token according to the preset token generation rate; if it is not necessary to regenerate the token, step 612 is executed to continue the token consumption logic.

[0155] After the token consumption is completed, the corresponding variable Key in Redis needs to be updated to refresh its token quantity and time information. This time information is used to determine whether to regenerate the token later.

[0156] Step 614: Return the remaining tokens and the waiting time.

[0157] Specifically, if the current token balance is sufficient, it returns the number of remaining tokens and determines whether the traffic requirement is met; at the same time, it returns information on whether a waiting time is required. This time information is used for subsequent logical judgments of the Redis access switch, i.e., controlling the state of the Redis access switch.

[0158] Once traffic returns to the pod, the newly generated token is stored in the local memory token bucket, and the timestamp information of the Redis access switch is updated. The traffic then continues to perform subsequent operations according to the established process.

[0159] For example, Redis is configured to generate tokens at the beginning of each second (e.g., 1 second can be divided into 1000 moments) at a preset token generation rate (e.g., 475 tokens per second). These generated tokens are temporarily stored in Redis. When traffic (e.g., 300 requests) accesses Redis through the Redis access switch, the current timestamp (e.g., the 100th moment in a 1-second period) is inconsistent with the preset timestamp (the 1st moment in a 1-second period). In this case, no token generation is needed; instead, the token consumption process begins (300 tokens are consumed from the 475 tokens), and the remaining 175 tokens are stored in the in-memory token bucket. The preset timestamp is then updated to the beginning of the next 1-second period, and the Redis access switch is turned off until the beginning of the next second, at which point it is turned on again. The traffic (300 requests) continues with subsequent operations.

[0160] See Figure 7 , Figure 7 A system rate limiting and processing architecture diagram according to one embodiment of this specification is shown. Figure 7 As shown, only the server-side is involved, not the client-side. The server-side comprises two key components: a business gateway and RPC services. The RPC services include the order placement service and other RPC services they depend on; these RPC services are treated as being at the same logical layer and described uniformly.

[0161] The system employs a unitized architecture, deploying a business gateway (730) and an RPC service in two regions: city A (710) and city B (720), forming two equal units. When a user initiates an order request (740), the system routes the request to the corresponding fixed unit based on the hash calculation result of the user's identifier (ID), and completes the entire order processing flow within that unit.

[0162] The business gateway will rate-limit the order placement requests: the business gateway has multiple container pods (750), and the multiple pods form a cluster. The pod cluster in the business gateway obtains tokens in batches from the database Redis (760), and then rate-limits the received order placement requests according to the tokens in the token bucket of the pod. The order placement requests that have obtained tokens will be forwarded by the business gateway to the downstream order placement RPC service 770.

[0163] In the order placement RPC service stage, RPC cluster rate limiting is implemented. Multiple pods (750) form a cluster. The pod cluster retrieves tokens in batches from the Redis database (760). Then, based on the tokens in the pod's token bucket, the order requests forwarded by the business gateway are rate-limited again. Order requests that re-acquire tokens will then make internal service calls through the order placement RPC service. Simultaneously, downstream RPC services are called concurrently. The RPC services that order placement depends on (780) include at least marketing (790), inventory (7100), shopping cart (7110), and address (7120) RPC services. After all RPC services have finished processing, the order database (7130) is updated. Order databases within different units need to be synchronized bidirectionally. The business gateways and RPC services in different units can share the Redis database or use their respective unit-specific Redis databases. This completes the order request processing flow.

[0164] In some embodiments, API rate limiting configurations are shown in Table 1.

[0165] Table 1 API Rate Limiting Configuration Table

[0166]

[0167] In the final configuration, the number of tokens generated per second is 475, the allowed caching time window is 5 seconds, which means the token bucket can hold a maximum of 475*5=2375 tokens, and the maximum number of orders processed per second is 2375.

[0168] In some embodiments, the RPC server thread pool configuration is shown in Table 2.

[0169] Table 2 RPC Server Thread Pool Configuration Table

[0170] Thread pool name Core thread count Maximum number of threads Configuration A thread pool 500 500 Detailed configuration

[0171] The core configuration items include: core thread count, maximum thread count, task queue size, and rejection policy. The core thread count equals the maximum thread count. The specific values ​​are determined based on the actual business situation. The default value for the RPC middleware is 500. The task queue size is 0, and the rejection policy is to discard tasks directly (AbortPolicy).

[0172] The effects of this configuration are as follows: 1. Fixed-size thread pool: Since the core thread count equals the maximum thread count, the thread pool size is fixed, always maintaining 500 running threads. Even idle threads will not be reclaimed (unless `lowCoreThreadTimeOut(true)` is set). 2. No task queue: The task queue size is 0, indicating the use of a zero-capacity synchronous queue. Tasks will not enter the queue to wait but will be directly submitted to the thread for execution. If there are no idle threads, a new task will trigger the rejection policy. 3. Rejection policy is AbortPolicy: When all threads in the thread pool are busy and cannot accept new tasks, a `RejectedExecutionException` will be thrown directly. This is a strict rejection policy, suitable for scenarios where explicit handling of task submission failures is required. 4. Execution flow: When a task is submitted: If the number of currently running threads is less than 500, the thread pool will create a new thread to execute the task. If the number of currently running threads has reached 500, the task will be directly rejected, and a `RejectedExecutionException` will be thrown.

[0173] In some embodiments, the RPC service rate limiting configuration is shown in Table 3.

[0174] Table 3 RPC Service Rate Limiting Configuration Table

[0175]

[0176] Table 3 RPC Service Rate Limiting Configuration Table (Continued)

[0177]

[0178] The RPC service uses cluster-based rate limiting with a threshold of 278 and a limiting window of 5 seconds. This means the RPC service's token bucket can hold a maximum of 278 * 5 = 1390 tokens, allowing a maximum of 1390 requests per second. Cluster rate limiting is a traffic control mechanism in distributed systems used to coordinate traffic distribution across multiple machines, preventing uneven resource allocation or system crashes due to performance differences between individual machines. Cluster rate limiting manages token generation and consumption across the entire cluster, rather than relying on independent calculations on individual machines. "Online promotion" refers to online promotion through various channels.

[0179] Figure 8 A schematic diagram illustrating a QPS variation provided in an embodiment of this specification is shown. For example... Figure 8 The graph shows the QPS curve over time, including four types of curves: rate limiting value, number of orders placed during live stream, number of orders placed outside of live stream, and total number of orders. The horizontal axis represents time, and the vertical axis represents quantity. The total number of orders is the sum of the number of orders placed during live stream and the number of orders placed outside of live stream. The graph shows that the peak QPS for orders was 840, and in actual testing, the maximum throughput per second was 835.

[0180] Figure 8 The corresponding RT change curve is Figure 9 , Figure 9 A schematic diagram of an RT variation provided in an embodiment of this specification is shown. Figure 9 The graph shows the RT (Response Time) curve over time, including two types of curves: non-live and live. The horizontal axis represents time, and the vertical axis represents the response time RT. As can be seen from the graph, for non-live orders, the pre-pulse RT is 302, the maximum RT during the pulse phase is 333, and the percentage of the maximum RT during the pulse phase to the pre-pulse RT is 110%. For live orders, the pre-pulse RT is 175, the maximum RT during the pulse phase is 390, and the percentage of the maximum RT during the pulse phase to the pre-pulse RT is 223%.

[0181] Figure 10 This diagram illustrates another QPS variation provided by an embodiment of this specification. For example... Figure 10 The graph shows the QPS curve over time, including four types of curves: rate limiting value, number of orders placed during live stream, number of orders placed outside of live stream, and total number of orders. The horizontal axis represents time, and the vertical axis represents quantity. The total number of orders is the sum of the number of orders placed during live stream and the number of orders placed outside of live stream. The graph shows that the peak QPS for orders was 1260, and in actual testing, the maximum throughput per second was 1196.

[0182] Figure 10 The corresponding RT change curve is Figure 11 , Figure 11 This diagram illustrates another RT variation provided in an embodiment of this specification. For example... Figure 11 The graph shows the response time (RT) as a function of time, including two types of curves: non-live and live. The horizontal axis represents time, and the vertical axis represents the response time (RT). As can be seen from the graph, for non-live orders, the pre-pulse RT is 288, the maximum RT during the pulse phase is 392, and the percentage of the maximum RT during the pulse phase to the pre-pulse RT is 136%. For live orders, the pre-pulse RT is 131, the maximum RT during the pulse phase is 392, and the percentage of the maximum RT during the pulse phase to the pre-pulse RT is 299%.

[0183] Figure 12 This diagram illustrates another QPS variation provided by an embodiment of this specification. For example... Figure 12 The graph shows the QPS curve over time, including four types of curves: rate limiting value, number of orders placed during live stream, number of orders placed outside of live stream, and total number of orders. The horizontal axis represents time, and the vertical axis represents quantity. The total number of orders is the sum of the number of orders placed during live stream and the number of orders placed outside of live stream. The graph shows that the peak QPS for orders was 1680, and in actual testing, the maximum throughput per second was 1579.

[0184] Figure 12 The corresponding RT change curve is Figure 13 , Figure 13 This diagram illustrates another RT variation provided in an embodiment of this specification. For example... Figure 13 The graph shows the response time (RT) as a function of time, including two types of curves: non-live and live. The horizontal axis represents time, and the vertical axis represents the response time (RT). As can be seen from the graph, for non-live orders, the pre-pulse RT is 266, the maximum RT during the pulse phase is 250, and the percentage of the maximum RT during the pulse phase to the pre-pulse RT is 94%. For live orders, the pre-pulse RT is 200, the maximum RT during the pulse phase is 253, and the percentage of the maximum RT during the pulse phase to the pre-pulse RT is 127%.

[0185] Figure 14 This diagram illustrates another QPS variation provided by an embodiment of this specification. For example... Figure 14 The graph shows the QPS curve over time, including four types of curves: rate limiting value, number of orders placed during live stream, number of orders placed outside of live stream, and total number of orders. The horizontal axis represents time, and the vertical axis represents quantity. The total number of orders is the sum of the number of orders placed during live stream and the number of orders placed outside of live stream. The graph shows that the peak QPS for orders was 2100, and in actual testing, the maximum throughput per second was 2026.

[0186] Figure 14 The corresponding RT change curve is Figure 15 , Figure 15 This diagram illustrates another RT variation provided in an embodiment of this specification. For example... Figure 15 The graph shows the response time (RT) as a function of time, including two types of curves: non-live and live. The horizontal axis represents time, and the vertical axis represents the response time (RT). As can be seen from the graph, for non-live orders, the pre-pulse RT is 240, the maximum RT during the pulse phase is 246, and the percentage of the maximum RT during the pulse phase to the pre-pulse RT is 103%. For live orders, the pre-pulse RT is 191, the maximum RT during the pulse phase is 257, and the percentage of the maximum RT during the pulse phase to the pre-pulse RT is 135%.

[0187] Figure 16This diagram illustrates another QPS variation provided by an embodiment of this specification. For example... Figure 16 The graph shows the QPS curve over time, including four types of curves: rate limiting value, number of orders placed during live stream, number of orders placed outside of live stream, and total number of orders. The horizontal axis represents time, and the vertical axis represents quantity. The total number of orders is the sum of the number of orders placed during live stream and the number of orders placed outside of live stream. The graph shows that the peak QPS for orders was 2520, and in actual testing, the maximum throughput per second was 2406.

[0188] Figure 16 The corresponding RT change curve is Figure 17 , Figure 17 This diagram illustrates another RT variation provided in an embodiment of this specification. For example... Figure 17 The graph shows the RT (Response Time) curve over time, including two types of curves: non-live and live. The horizontal axis represents time, and the vertical axis represents the response time RT. As can be seen from the graph, for non-live orders, the pre-pulse RT is 240, the maximum RT during the pulse phase is 340, and the percentage of the maximum RT during the pulse phase to the pre-pulse RT is 140%; for live orders, the pre-pulse RT is 190, the maximum RT during the pulse phase is 380, and the percentage of the maximum RT during the pulse phase to the pre-pulse RT is 200%.

[0189] Figure 18 This diagram illustrates another QPS variation provided by an embodiment of this specification. For example... Figure 18 The graph shows the QPS curve over time, including four types of curves: number of orders triggered by traffic throttling, number of orders placed during live stream, number of orders placed outside of live stream, and total number of orders. The horizontal axis represents time, and the vertical axis represents quantity. The total number of orders is the sum of the number of orders placed during live stream and the number of orders placed outside of live stream. The graph shows that during the flash sale, at 22:00:01, the highest number of orders placed per second (2114) did not trigger traffic throttling. The number of orders placed during live stream was 29, and the number of orders placed outside of live stream was 2085. The number of orders triggered by traffic throttling was 0, meaning that 475 orders per second were used to handle 2114 orders per second of flash sale traffic.

[0190] In the embodiments of this specification, (1) the design concept is simple and straightforward, without using complex algorithms, and the required functions can be achieved only through a mature and reliable token bucket rate limiting model and thread pool configuration. This reduces the implementation difficulty and maintenance cost, and shortens the development and deployment cycle; under the premise of meeting the high availability requirements, it avoids the system stability risk caused by the introduction of excessively complex logic. (2) The gateway layer and RPC service entry point uniformly adopt token bucket rate limiting, and the number of tokens generated per second and the maximum number of tokens accumulated in the token bucket are set according to different business scenarios and resource characteristics. Without additional expansion of hardware / computing resources, it can smoothly absorb short-term traffic surges (pulses), avoid instantaneous request peaks causing a sudden increase in backend service pressure, and prevent system jitter or unavailability, thereby improving the overall throughput and availability. (3) The RPC server thread pool is configured with a "prevent thread queuing" strategy. When the thread pool is full, it adopts methods such as direct rejection (RejectedExecutionHandler is Abort) to prevent requests from queuing in the thread queue for a long time. It avoids the risk of uncontrollable service response time (RT) caused by thread accumulation; when the system reaches its bottleneck, it can quickly respond instead of waiting indefinitely, reducing the possibility of backend resource exhaustion caused by excessively long queues, thereby ensuring that the system can maintain stable and predictable response performance in high-concurrency scenarios. (4) It is seamlessly integrated into the existing microservice architecture. The implementation can be directly deployed in the existing gateway and RPC framework without the need to introduce additional third-party rate limiting middleware or complex components. It integrates into the existing system architecture at zero / low cost, reducing the scope of transformation of the online environment; it avoids the risk of large-scale technology stack upgrades and accelerates the implementation speed.

[0191] Corresponding to the above method embodiments, this specification also provides embodiments of a task processing device. Figure 19 A schematic diagram of a task processing apparatus according to one embodiment of this specification is shown. Figure 19 As shown, the task processing device includes: an acquisition module 1910 and a processing module 1920;

[0192] The acquisition module 1910 is configured to acquire tokens from a token vault of a specified capacity in response to a task processing request.

[0193] The processing module 1920 is configured to call pre-allocated callable threads in the thread pool to process the target task processing request for which a token has been obtained, and to obtain the task processing result. The thread pool has a specified number of threads pre-allocated.

[0194] In some embodiments, the acquisition module 1910 is further configured to generate a token at a preset rate and store the generated token in a token vault.

[0195] In some embodiments, the processing module 1920 is also configured to create a specified number of threads and build a thread pool based on the threads.

[0196] In some embodiments, the target task processing request includes multiple concurrent target task processing requests; the processing module 1920 is further configured to, for the multiple target task processing requests that have obtained a token, if the number of requests of the multiple target task processing requests is greater than the number of threads that can be called, determine the task processing requests to be rejected from the multiple target task processing requests and refuse to process the task processing requests to be rejected.

[0197] In some embodiments, the processing module 1920 is further configured to refuse to process initial task processing requests for which no token has been obtained.

[0198] In some embodiments, the processing module 1920 is further configured to allocate a token from the tokens to be stored for an initial task processing request that has not obtained a token, and add the remaining tokens to be stored to the token library, wherein the tokens to be stored are tokens generated by the token generation unit but not added to the token library, and the remaining tokens to be stored are tokens to be stored that have not been allocated to the initial task processing request.

[0199] In some embodiments, the processing module 1920 is further configured to obtain the access status of the token generation unit; if the access status is allowed, allocate a token for the initial task processing request from the tokens to be stored, and add the remaining tokens to be stored to the token library.

[0200] In some embodiments, the processing module 1920 is further configured to generate a token to be stored at a preset rate if the current timestamp and the preset time point meet the token generation conditions.

[0201] Figure 20 A structural block diagram of a computing device 2000 provided according to one embodiment of this specification is shown.

[0202] The computing device 2000 includes:

[0203] The memory 2010 and the processor 2020 are connected via a bus 2030;

[0204] The memory 2010 is used to store computer programs / instructions, and the processor 2020 is used to execute the computer programs / instructions, which implement the steps of the method when executed by the processor 2020.

[0205] Specifically, the components of the computing device 2000 include, but are not limited to, a memory 2010 and a processor 2020. The processor 2020 is connected to the memory 2010 via a bus 2030, and the database 2050 is used to store data.

[0206] The computing device 2000 also includes an access device 2040, which enables the computing device 2000 to communicate via one or more networks 2060. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 2040 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0207] In one embodiment of this specification, the above-described components of the computing device 2000 and Figure 20 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 20 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0208] The computing device 2000 can be any type of stationary or mobile electronic device, including mobile computers or mobile electronic devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable electronic devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary electronic devices such as desktop computers or personal computers (PCs). The computing device 2000 can also be a mobile or stationary server.

[0209] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the task processing method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the task processing method described above.

[0210] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the video generation method, video generation model training method, virtual character video generation method, and model training method described above.

[0211] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solutions of the video generation method, video generation model training method, virtual character video generation method, and model training method described above. For details not described in detail in the technical solution of the storage medium, please refer to the descriptions of the technical solutions of the video generation method, video generation model training method, virtual character video generation method, and model training method described above.

[0212] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described video generation method, video generation model training method, virtual character video generation method, and model training method.

[0213] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product belongs to the same concept as the technical solutions of the video generation method, video generation model training method, virtual character video generation method, and model training method described above. For details not described in detail in the technical solution of the computer program product, please refer to the descriptions of the technical solutions of the video generation method, video generation model training method, virtual character video generation method, and model training method described above.

[0214] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0215] Computer programs / instructions include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. Computer-readable media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in computer-readable media can be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0216] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0217] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0218] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to specific implementations. Clearly, many modifications and variations can be made based on the embodiments described in this specification. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A task processing method, characterized in that, The method includes: In response to a task processing request, obtain a token from a token vault of a specified size; For a target task processing request that has obtained the token, a pre-allocated callable thread in the thread pool is invoked to process the target task processing request and obtain the task processing result. The thread pool has a specified number of pre-allocated threads.

2. The method according to claim 1, characterized in that, Before retrieving a token from a token vault of a specified capacity in response to a task processing request, the method further includes: Tokens are generated according to a preset rate and stored in the token vault.

3. The method according to claim 1, characterized in that, Before retrieving a token from a token vault of a specified capacity in response to a task processing request, the method further includes: Create a specified number of the threads, and build the thread pool based on the threads.

4. The method according to claim 1, characterized in that, The target task processing request includes multiple concurrent target task processing requests; After retrieving a token from a token vault of a specified capacity in response to a task processing request, the method further includes: For the multiple target task processing requests that have obtained tokens, if the number of multiple target task processing requests is greater than the number of callable threads, the task processing requests to be rejected are determined from the multiple target task processing requests, and the task processing requests to be rejected are rejected.

5. The method according to claim 1, characterized in that, After retrieving a token from a token vault of a specified capacity in response to a task processing request, the method further includes: For initial task processing requests that have not obtained a token, the initial task processing request will be rejected.

6. The method according to any one of claims 1 to 5, characterized in that, After retrieving a token from a token vault of a specified capacity in response to a task processing request, the method further includes: For an initial task processing request that has not obtained a token, a token is allocated to the initial task processing request from the tokens to be stored, and the remaining tokens to be stored are added to the token library. The tokens to be stored are tokens generated by the token generation unit but not added to the token library, and the remaining tokens to be stored are the tokens to be stored that have not been allocated to the initial task processing request.

7. The method according to claim 6, characterized in that, Before allocating a token for the initial task processing request from the tokens to be stored and adding the remaining tokens to the token vault, the method further includes: Obtain the access status of the token generation unit; The step of allocating a token for the initial task processing request from the tokens to be stored, and adding the remaining tokens to the token vault, includes: If the access status is allowed, a token is allocated for the initial task processing request from the tokens to be stored, and the remaining tokens to be stored are added to the token library.

8. The method according to claim 6, characterized in that, Before allocating a token for the initial task processing request from the tokens to be stored and adding the remaining tokens to the token vault, the method further includes: If the current timestamp and the preset time point meet the token generation conditions, the token to be stored is generated according to the preset rate.

9. A task processing system, characterized in that, Includes the rate limiting layer and the service processing layer; The rate limiting layer is used to obtain tokens from a token vault of a specified capacity in response to a task processing request. The service processing layer is used to call pre-allocated callable threads in the thread pool to process the target task processing request that has obtained the token, and to obtain the task processing result. The thread pool has a specified number of threads pre-allocated.

10. The system according to claim 9, characterized in that, The task processing system also includes a business gateway and service components; The rate limiting layer is deployed on the business gateway and the service component, respectively; The service processing layer is deployed on the service component.

11. The system according to claim 9 or 10, characterized in that, The task processing system also includes a routing layer, the rate limiting layer includes multiple rate limiting units with different deployment information, and the service processing layer includes multiple service processing units with different deployment information. The routing layer is used to select the target rate limiting unit and the target service processing unit from the plurality of rate limiting units and the plurality of service processing units according to the user identifier corresponding to the task processing request. The target rate limiting unit is used to obtain the token from the token vault of a specified capacity in response to the task processing request; The target service processing unit is used to call a pre-allocated callable thread in the thread pool to process the target task processing request that has obtained the token, and to obtain the task processing result. The thread pool has a specified number of threads pre-allocated.

12. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 8.

13. A computer-readable storage medium, characterized in that, It stores computer-executable instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 8.

14. A computer program product, characterized in that, Includes a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 8.

Images