Request processing method and apparatus, electronic device, computer-readable storage medium, and computer program product

Abstract

The application provides a request processing method and device, electronic equipment, computer readable storage medium and computer program product. The method comprises the following steps: in response to receiving a first request sent by a first channel in a first period, determining a first total number of requests received in the first period under the condition that the flow state of the first period is an unflow limiting state; in the case that the first total number is less than a first flow threshold and greater than or equal to a second flow threshold, adjusting the flow state from the unflow limiting state to a flow limiting state; limiting the number of second requests sent by the first channel and received in the first period based on a third flow threshold corresponding to the first channel, the sending time of the second request being located after the sending time of the first request. Through the application, the flow limiting strategy can be flexibly configured, and the availability and stability of the system are improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a request processing method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] With the rapid development of internet technology, software systems face increasingly high concurrent access pressure. To ensure the high availability of application services and prevent performance issues caused by excessive request traffic, rate limiting technology is widely used in software systems. Rate limiting refers to controlling the number of requests within a certain period of time to ensure the stable operation of the software system, thereby achieving high availability of application services.

[0003] The rate limiting methods in related technologies are usually comprehensive rate limiting or partial rate limiting. Comprehensive rate limiting applies a uniform rate limiting strategy to all callers, while partial rate limiting applies a fixed rate limiting strategy to a single caller. In the case of multiple callers, these methods have relatively simple rate limiting strategies and are difficult to configure flexibly, resulting in the need to improve system availability and stability. Summary of the Invention

[0004] This application provides a request processing method, apparatus, computer-readable storage medium, and computer program product, which can flexibly configure rate limiting strategies to improve system availability and stability.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a request processing method, the method comprising:

[0007] In response to receiving a first request from a first channel during a first time period, if the traffic status during the first time period is unrestricted, a first total number of requests received during the first time period is determined; if the first total number is less than a first traffic threshold but greater than or equal to a second traffic threshold, the traffic status is adjusted from unrestricted to restricted; based on a third traffic threshold corresponding to the first channel, the number of second requests received from the first channel during the first time period is limited, wherein the sending time of the second request is after the sending time of the first request.

[0008] This application provides a request processing apparatus, including:

[0009] The request quantity determination module is used to determine the first total number of requests received in the first time period in response to receiving a first request sent by a first channel in the first time period, provided that the traffic status in the first time period is unlimited.

[0010] The status adjustment module is used to adjust the flow status from the unlimited state to the limited state when the first total quantity is less than the first flow threshold and greater than or equal to the second flow threshold.

[0011] The rate limiting module is used to limit the number of second requests sent by the first channel within the first time period based on the third traffic threshold corresponding to the first channel, wherein the sending time of the second request is after the sending time of the first request.

[0012] This application provides an electronic device, the electronic device comprising:

[0013] Memory is used to store executable instructions or computer programs.

[0014] The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the request processing method provided in the embodiments of this application.

[0015] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the request processing method provided in this application when executed by a processor.

[0016] This application provides a computer program product, including a computer program or computer executable instructions, which, when executed by a processor, implements the request processing method provided in this application.

[0017] The embodiments of this application have the following beneficial effects:

[0018] In this manner, when a first request is received from the first channel within the first time period, if the traffic status within the first time period is unrestricted, the total number of requests received within the first time period can be compared with a first traffic threshold and a second traffic threshold. If the total number is less than the first traffic threshold but greater than or equal to the second traffic threshold, the traffic status is triggered to change from unrestricted to restricted. Furthermore, the number of second requests sent by the first channel within the first time period is limited based on the third traffic threshold corresponding to each first channel. This method, through the dual threshold judgment of the first and second traffic thresholds, uses the second traffic threshold as the trigger for traffic restriction. The system uses a warning threshold to activate rate limiting in advance when the current traffic is close to but has not yet reached the first traffic threshold (i.e., the first total number is greater than or equal to the second traffic threshold, but less than the first traffic threshold). This prevents the system from becoming overloaded. Furthermore, the system can flexibly decide whether to activate rate limiting based on the setting of the second traffic threshold. In rate limiting mode, the number of requests is limited based on the threshold corresponding to the first channel, enabling flexible configuration of traffic control for the first channel. This allows for finer-grained traffic control of the first channel, ensuring high system availability. In addition, the second traffic threshold provides a buffer zone for the system, ensuring system stability. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the architecture of the request processing system provided in the embodiments of this application;

[0020] Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0021] Figure 3 This is a first flowchart illustrating the request processing method provided in this application embodiment;

[0022] Figure 4A This is a flowchart illustrating the current limiting method of the first current limiter provided in this application embodiment;

[0023] Figure 4B This is a flowchart illustrating the current limiting method of the second current limiter provided in an embodiment of this application;

[0024] Figure 5 This is a second flowchart illustrating the request processing method provided in the embodiments of this application.

[0025] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0028] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0029] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0030] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0031] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0032] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0033] Response: Used to indicate the conditions or states on which the operation performed depends. When the conditions or states on which it depends are met, one or more operations performed can be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which multiple operations are performed.

[0034] This application provides a request processing method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can flexibly adjust rate limiting strategies and improve system availability and stability.

[0035] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of the request processing system 100 provided in the embodiments of this application. In order to support a request processing application, the terminal (terminal 400-1 and terminal 400-2 are shown as examples) connects to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0036] In this embodiment, the first channel can represent a caller, i.e., the entity that initiates the request. Different first channels can represent different callers, or callers of different types. When the first channel represents different callers, one first channel usually represents one caller; when the first channel represents callers of different types, one first channel usually includes multiple callers of the same type.

[0037] Here, a terminal (such as terminal 400-1, terminal 400-2) can be regarded as a primary channel. Different terminals (such as terminal 400-1, terminal 400-2) send requests to the software systems deployed on server 200 as different calling directions to invoke the corresponding services.

[0038] In some embodiments, after any terminal (such as terminal 400-1 or terminal 400-2) sends a first request to server 200 within a first time period, server 200 may, in response to receiving the first request sent by the first channel within the first time period, determine the first total number of requests received within the first time period if the traffic status of the first time period is unlimited; if the first total number is less than a first traffic threshold and greater than or equal to a second traffic threshold, adjust the traffic status from unlimited to limited; and based on a third traffic threshold corresponding to the first channel, limit the number of second requests received from the first channel within the first time period, wherein the sending time of the second request is after the sending time of the first request.

[0039] In some embodiments, the terminal can be implemented as various types of terminals such as laptops, tablets, desktop computers, set-top boxes, smartphones, smart speakers, smartwatches, smart TVs, and in-vehicle terminals, or it can be implemented as a server.

[0040] In some embodiments, server 200 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminals and servers can be connected directly or indirectly via wired or wireless communication, which is not limited in this embodiment.

[0041] See Figure 2 , Figure 2 This is a schematic diagram of the structure of the electronic device 400 provided in the embodiments of this application. Figure 2 The illustrated electronic device 400 includes at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in the electronic device 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 440.

[0042] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0043] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0044] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0045] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0046] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0047] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0048] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0049] Presentation module 453 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 (e.g., a display screen, a speaker, etc.) associated with user interface 430;

[0050] The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0051] In some embodiments, the request processing apparatus provided in this application can be implemented in software. Figure 2 A request processing device 455 stored in memory 450 is shown. This device can be software in the form of programs or plug-ins, and includes the following software modules: a request quantity determination module 4551, a status adjustment module 4552, and a rate limiting module 4553. These modules are logically linked and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0052] In other embodiments, the request processing apparatus provided in this application can be implemented in hardware. As an example, the request processing apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the request processing method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0053] In some embodiments, the terminal or server can implement the request processing method provided in this application by running various computer-executable instructions or computer programs. For example, computer-executable instructions can be microprogram-level commands, machine instructions, or software instructions. Computer programs can be native programs or software modules in an operating system; they can be native applications (APPs), i.e., programs that need to be installed in the operating system to run, such as instant messaging APPs; or they can be applets that can be embedded in any APP, i.e., programs that only need to be downloaded to a browser environment to run. In summary, the aforementioned computer-executable instructions can be any form of instruction, and the aforementioned computer programs can be any form of application, module, or plugin.

[0054] The request processing method provided in the embodiments of this application will now be described in conjunction with the accompanying drawings. As mentioned above, the electronic device 400 implementing the request processing method of the embodiments of this application can be a terminal, a server, or a combination of both. Therefore, the executing entity of each step will not be described again below.

[0055] The request processing method of this application embodiment will be described using a server as an example of the executing entity. See also Figure 3 , Figure 3 This is a first flowchart illustrating the request processing method provided in this application embodiment, which will be combined with... Figure 3 The steps shown are explained.

[0056] In step 101, in response to receiving a first request sent by a first channel in a first time period, and if the traffic status in the first time period is unlimited, a first total number of requests received in the first time period is determined.

[0057] Here, the number of primary channels can be one or more (greater than or equal to two). A primary channel can represent a caller, that is, the entity that initiates the request. Different primary channels can represent different callers, or callers of different types. When primary channels represent different callers, one primary channel usually represents one caller; when primary channels represent callers of different types, one primary channel usually includes multiple callers of the same type.

[0058] In this context, the caller refers to the client or server entity that initiates a request to the software system. These entities can be different business systems, applications, user devices, or other service modules. Different callers, or different types of callers, may have different requirements regarding system resource usage and request frequency based on their business needs and priorities.

[0059] As an example, the caller is the entity that initiates the request in the request processing system. The caller can be: an external system, such as a third-party application or partner system; an internal service, such as other service modules in a microservice architecture; a user device, such as a mobile device, web browser, or other terminal device; or a specific business module, such as an order service or payment service within the system. In other words, in Figure 1 In the request processing system shown, each terminal can act as the subject initiating the request, that is, as the caller.

[0060] It should be noted that the software system proposed here is the entity that implements the request processing method of this application embodiment. The software system is deployed in server 200. In this process, the software system acts as the callee, receiving and processing requests sent from the first channel, and is responsible for traffic control, resource allocation, and response processing of the requests sent from the first channel. The callee typically has the following characteristics: a request receiver, receiving requests from the caller (i.e., the first channel) and processing them; a resource provider, providing computing, storage, network, and other resources to complete the request processing; a rate limiting implementer, controlling requests through rate limiting mechanisms to prevent system overload; and a service provider, providing specific business functions or services to the caller, such as data querying, transaction processing, and computing tasks.

[0061] Understandably, the interaction between the caller and the callee can include the following: the callee receives a request from the caller and parses the request content (such as request parameters); when the callee receives a request from the caller, it checks whether the number of requests exceeds the threshold; it performs business logic processing on valid requests and generates a response result; it returns the processing result to the caller, while recording the request status and performance metrics; and it rejects or queues requests that exceed the rate limiting threshold and returns the corresponding error information.

[0062] In practice, the first time period can be understood as a time window. In rate limiting strategies, a time window refers to a fixed period of time used to measure and limit the number of requests. Specifically, the time window is used to count the total number of requests received by the software system (i.e., the callee, which can also be regarded as the server) within a specific time period, and to determine whether to allow or deny subsequent requests based on a preset threshold.

[0063] As an example, the first time period is usually set to a fixed time interval, such as 1 second, 1 minute, 1 hour, etc. The specific first time period can be set according to actual needs.

[0064] For example, assuming the first time window is set to 1 second, if the maximum queries per second (QPS) of the called party is 100, then the software system can handle a maximum of 100 requests within each 1-second time window. If the number of requests exceeds 100, subsequent requests will be rate-limited. Assuming the first time window is set to 1 minute, if the maximum requests per minute (RPM) of the called party is 6000, then the software system can handle a maximum of 6000 requests within each 1-minute time window. If the number of requests exceeds 6000, subsequent requests will be rate-limited.

[0065] It should be noted that short time windows (e.g., 1 second) are suitable for scenarios requiring fine-grained control of request rates, such as interface rate limiting and preventing sudden traffic surges; long time windows (e.g., 1 minute, 1 hour) are suitable for scenarios with less stringent request rate requirements, such as user behavior analysis and resource usage statistics. The duration of the first time period can be set based on actual needs and is not specifically limited here. Generally, after determining the duration of the detection time window, the first and second time periods are treated as different time windows with the same duration. For example, if the time window is set to 1 second, then the duration of both the first and second time periods is 1 second.

[0066] The first time period, serving as a time window, can be either sliding or fixed. A sliding time window continuously updates the statistical range over time, while a fixed time window resets the statistics at the end of each time window. In this embodiment, the first time period is a fixed time window, and the statistics are reset at the end of the first time period, with the number of requests being recounted in the next time period (e.g., the second time period).

[0067] Typically, the callee (i.e., the software system in the server) is configured with a rate limiter, which executes the request processing method of this application embodiment. A rate limiter is a mechanism used to control the request rate or resource usage, designed to prevent the system from crashing due to overload and ensure the stability and availability of the software system. The rate limiter protects the system from sudden traffic, malicious attacks, or resource abuse by limiting the number of requests or resource usage within a unit time window.

[0068] In actual implementation, the rate limiter for the called party is also configured with a traffic status, which includes two states: unlimited and limited. The unlimited state is the default state, which is set as a normal state. When the rate limiter starts, the traffic status of the rate limiter is unlimited. When the number of requests within the time window increases to the point of triggering a certain threshold, the traffic status will be triggered and adjusted to the limited state. When the state adjustment condition is met, it will return to the unlimited state, that is, return to the normal state.

[0069] Therefore, if the first time period is the current time window for request quantity detection, in response to receiving a first request from any first channel within the first time period, the traffic status of the first time period is first determined. If the traffic status of the first time period is unlimited, the first total number of requests received within the first time period is determined. Here, if there is only one first channel, the first total number is the total number of all requests (including the first request) sent by that first channel received within the first time period; that is, the received requests include all requests (including the first request) sent by that first channel within the first time period. If there are multiple first channels, the first total number is the total number of all requests (including the first request) sent by all first channels received within the first time period; that is, the received requests include all requests (including the first request) sent by all first channels within the first time period.

[0070] Then, based on the first total number, it is determined whether traffic limiting is necessary.

[0071] In some embodiments, after determining the first total number of requests received in the first time period, the server may limit the number of requests by stopping receiving second requests sent through the first channel if the first total number exceeds a first traffic threshold.

[0072] In practice, the rate limiters configured for the callee include a first rate limiter, a second rate limiter, and a third rate limiter. The first rate limiter is used to limit the maximum number of requests that the callee can receive within a time window. The second rate limiter is used to limit the number of requests that the callee can receive within a time window to a healthy state (i.e., a reasonable state). The third rate limiter is configured for different channels, and different first channels correspond to different third rate limiters. The third rate limiter is used to limit the number of requests that can be received from the corresponding channel within a time window.

[0073] Here, we will first explain the first current limiter.

[0074] The first rate limiter is configured with a first traffic threshold, which can be understood as the upper limit of the number of requests that the callee can receive within a time window.

[0075] Figure 4A This is a flowchart illustrating the current limiting method of the first current limiter provided in this application embodiment. See also... Figure 4A The current limiting process of the first current limiter includes:

[0076] S201, request to receive.

[0077] S202, determine whether to limit the flow.

[0078] Upon receiving a request, determine the first total number of requests received within the current time window. Determine whether the first rate limiter is triggered by checking if the first total number is greater than the first traffic threshold. If triggered, rate limiting is applied; otherwise, continue with subsequent processing.

[0079] S203, flow restriction.

[0080] If the first rate limiter is triggered, the entire callee will be rate-limited, meaning it will stop receiving all requests (i.e., requests will be rejected).

[0081] S204, continue to determine the status of other current limiters.

[0082] If the first rate limiter is not triggered, it will continue to determine whether other rate limiters (such as the second rate limiter and the third rate limiter) are triggered, and execute the subsequent rate limiting logic according to the other rate limiters.

[0083] Therefore, after determining the first total number of requests received within the first time period, the first total number is first compared with the first traffic threshold of the first flow limiter. If the first total number is greater than the first traffic threshold, it means that if requests are continued to be received, the software system will crash due to overload. Therefore, when it is determined that the first total number is greater than the first traffic threshold, the reception of the second request sent by the first channel is stopped directly.

[0084] Here, the second request is a request that follows the first request, that is, a request that is sent after the first request was sent through the first channel.

[0085] By setting a first traffic threshold, the system can immediately reject all subsequent requests once the total number of requests received within a first time period exceeds this threshold. This prevents the called party's resources from being exhausted due to a continuous influx of requests, thus avoiding overload and ensuring the stability and sustainable availability of the software system. Furthermore, this method can also quickly block requests during sudden traffic spikes (such as malicious attacks) to prevent cascading failures.

[0086] See also Figure 3 The following explanation follows step 101 above.

[0087] In step 102, if the first total quantity is less than the first flow threshold and greater than or equal to the second flow threshold, the flow status is adjusted from unrestricted to flow-restricted.

[0088] Here, we will explain the second current limiter.

[0089] The second flow limiter is configured with a second flow threshold, which is used to limit the reasonableness of the number of requests received by the callee within a time window. Generally, the second flow threshold is less than the first flow threshold.

[0090] In this embodiment of the application, whether the second flow limiter is triggered determines whether the flow state is adjusted from an unlimited state to a flow-limited state.

[0091] Figure 4B This is a flowchart illustrating the current limiting method of the second current limiter provided in this application embodiment. See also... Figure 4B The current limiting process of the second current limiter includes:

[0092] S301, request to receive.

[0093] S302, determine whether to limit the flow.

[0094] Upon receiving a request, determine the first total number of requests received within the current time window. Determine whether the second rate limiter is triggered by checking if the first total number is greater than or equal to the second traffic threshold. If triggered, execute the corresponding rate limiting strategy.

[0095] S303, Adjust flow status.

[0096] If the second rate limiter is triggered, the traffic status will be changed from unlimited to limited, and the corresponding rate limiting strategy will be used to limit the traffic.

[0097] S304 records the time when the traffic status was adjusted.

[0098] S305, no data throttling.

[0099] If the first total number is less than the second traffic threshold, it means that the number of requests received within the current time window is within a reasonable range, and there is no need to limit the flow; the received requests can be processed normally.

[0100] Therefore, after determining the first total number of requests received within the first time period and confirming that this first total number is less than the first traffic threshold of the first rate limiter, the first total number can be compared with the second traffic threshold of the second rate limiter. If the first total number is greater than or equal to the second traffic threshold, it indicates that although the first total number of requests received within the first time period has not yet reached the upper limit that the software system can accept, the first total number is already in a relatively high state. At this time, some rate limiting is required to ensure that the first total number does not exceed the first traffic threshold, thus preventing the software system from overloading. Therefore, when it is determined that the first total number is greater than or equal to the second traffic threshold, the traffic status is first adjusted from an unrated state to a rate-limited state, thereby starting to rate limit the requests within the first time period using the method described in step 103.

[0101] By employing the first and second traffic thresholds described above, finer-grained traffic control can be provided. Setting the second traffic threshold allows for timely limiting of subsequent requests when the initial total number of requests received within a first time period is high. This prevents overload and crashes caused by the initial total number exceeding the first traffic threshold. The second traffic threshold essentially provides a buffer zone for the system, giving it flexibility to adjust and respond to traffic management, ensuring the stability and availability of the software system. Furthermore, dynamically adjusting the first and second traffic thresholds allows the system to adapt to different environments, improving the flexibility of rate limiting.

[0102] See also Figure 3 The following explanation follows step 102 above.

[0103] In step 103, the number of second requests received from the first channel within the first time period is limited based on the third traffic threshold corresponding to the first channel.

[0104] The second request was sent after the first request.

[0105] Here, we will explain the third current limiter.

[0106] The third rate limiter is configured for the first channel. One first channel corresponds to one third rate limiter. Therefore, the ratio of the first, second, and third rate limiters can be 1:1:n, where n is a positive integer greater than or equal to 1. The third rate limiter is only responsible for limiting requests to its corresponding first channel, and it only executes the corresponding rate limiting policy for the first channel when the traffic status is in a rate-limited state.

[0107] A third flow threshold is configured for the third flow limiter. This third flow threshold can be flexibly set, allowing for different flow limiting strategies. When flow limiting is applied, the third flow threshold corresponding to the first channel is the same as the third flow threshold of the third flow limiter corresponding to the first channel.

[0108] In actual implementation, after the traffic status is adjusted from unlimited to limited, the corresponding first channel is limited by the third rate limiter. For each first channel, the corresponding third rate limiter and the corresponding third traffic threshold are determined. Based on the third traffic threshold, the third rate limiter limits the number of second requests sent by the first channel within the first time period. In other words, it limits the number of requests sent by the first channel after sending the first request.

[0109] In some embodiments, Figure 3 Step 103 can be implemented in the following way: if the fourth total number of first requests sent by the first channel received in the first time period is greater than or equal to the third traffic threshold, stop receiving second requests sent by the first channel in the first time period; if the fourth total number is less than the third traffic threshold, determine the difference between the first traffic threshold and the second traffic threshold, as well as the minimum value of the third traffic threshold and the difference, and control the fifth total number of second requests sent by the first channel received in the first time period to be less than the minimum value.

[0110] In practice, when there are multiple primary channels and no hierarchy among them, the third traffic threshold for each primary channel can be determined based on its call requirements. For example, the higher the call requirements of a primary channel, the higher the third traffic threshold can be configured for that channel.

[0111] In practice, when setting the third traffic threshold for all first channels, the total number generally does not exceed the first traffic threshold.

[0112] As an example, suppose there are three primary channels: System A, System B, and System C, with System D being the caller. System D can receive a maximum of 1000 requests within a time window. Therefore, the first traffic threshold can be set to 1000 QPS, and the second traffic threshold to 800 QPS. When setting the three third traffic thresholds for the three primary channels, the sum of the three third traffic thresholds can be set based on the call demands of each primary channel, ensuring that the sum of the three third traffic thresholds does not exceed the first traffic threshold. For example, if the call demand of System A is less than that of System B, and the call demand of System B is less than that of System C, then the third traffic threshold for System A can be set to 200 QPS, the third traffic threshold for System B to 300 QPS, and the third traffic threshold for System C to 500 QPS.

[0113] In actual implementation, taking the configuration in the example above as an example, assuming that the first total number of requests received in the first time period reaches 800, then when any first request sent by the first channel is received in the first time period, the first total number will be 801. If the first total number is determined to be greater than or equal to the second traffic threshold of 800 QPS, then the traffic status will be adjusted from the unlimited state to the limited state, and the number of requests for their respective first channels will be controlled by each third rate limiter, that is, the number will be limited.

[0114] Here, when limiting the number of requests for any first channel, it is first necessary to determine the fourth total number of first requests sent by the first channel that have been received in the first time period, and then compare the fourth total number with the corresponding third traffic threshold.

[0115] In one scenario, if the total number of requests received by the first channel within the first time period is greater than or equal to the corresponding third traffic threshold, it indicates that the number of requests received by the first channel within the first time period has reached its maximum set value. In this case, receiving second requests sent by the first channel will cease within the first time period. This cessation of reception can be manifested by rejecting or queuing the second requests sent by the first channel and returning an error message accordingly to the caller of the first channel.

[0116] In another scenario, if the fourth total number is less than the third traffic threshold, it indicates that the number of first requests sent by the first channel before rate limiting was reasonable and normal. Therefore, after rate limiting, only the number of second requests sent needs to be restricted; there's no need to stop receiving. Specifically, since the traffic state changes from unrestricted to rate-limited due to the triggering of the second rate limiter, the first total number of requests received in the first time period has at least reached the second traffic threshold. To ensure that the total number of requests received in the first time period does not exceed the first traffic threshold, the difference between the first and second traffic thresholds can be determined. Ensuring that the number of requests received after rate limiting does not exceed this difference will prevent system overload. For the first channel, the number of second requests sent by the first channel is also limited by the corresponding third traffic threshold. Therefore, the minimum value among the differences between the third, first, and second traffic thresholds is determined, and this minimum value is used to limit the fifth total number of second requests received from the first channel in the first time period—that is, ensuring the fifth total number is less than the minimum value.

[0117] As an example, suppose there are three primary channels: System A, System B, and System C, with System D being the caller. System D can receive a maximum of 1000 requests within a time window. Therefore, the first traffic threshold can be set to 1000 QPS, the second traffic threshold to 800 QPS, the third traffic threshold for System A to 200 QPS, the third traffic threshold for System B to 300 QPS, and the third traffic threshold for System C to 500 QPS.

[0118] In one example, during the first time period, if 10 first requests from system B and 10 first requests from system C have been received, system A suddenly experiences a traffic surge, sending 2000 first requests to system D. At this point, since the second traffic threshold is 800 QPS, and systems B and C have already occupied 20 requests, the number of first requests that can be normally received from system A is 779. When system A sends its 780th first request, the total number of first requests in the first time period reaches 800, which is greater than or equal to the second traffic threshold, triggering the second rate limiter. This causes the traffic status to change from unrestricted to rate-limited, and each third rate limiter begins to limit the traffic on its respective first channel. Here, since the fourth total number of first requests received from System A in the first time period is 781, which exceeds the third traffic threshold of 200 QPS for System A, receiving second requests from System A is stopped in the first time period. For System B, the fourth total number is 20, which is less than the third traffic threshold of 300 QPS for System B. Therefore, it is sufficient to ensure that the number of second requests received from System B in the first time period is less than the minimum value of 200. Similarly, it is also necessary to ensure that the number of second requests received from System C in the first time period is less than 200. In addition, it is also necessary to ensure that the sum of the number of second requests received from System B and System C in the first time period is less than 200. That is, in addition to ensuring that the fourth total number is less than the third traffic threshold and limiting the fifth total number of second requests received from the first channel in the first time period to be less than the minimum value, it is also necessary to limit the number of second requests received from all the first channels in the first time period to be less than the minimum value.

[0119] In one example, suppose that 200 first requests from system B and 300 first requests from system C have been received in the first time period. If system A suddenly experiences a traffic surge, sending 2000 first requests to system D, and the total first request count is 500, then the first 299 first requests from system A can be received normally. When system A sends its 300th first request, the total first request count in the first time period reaches 800, which is greater than or equal to the second traffic threshold, triggering the second rate limiter. This causes the traffic status to change from unlimited to rate-limited, and each third rate limiter begins to limit the traffic on its respective first channel. During rate limiting, receiving second requests from system A is stopped. The total fifth number of second requests that systems B and C can receive is less than 200, and the sum of the total fifth number of second requests received from system B and system C is also less than 200.

[0120] In one example, suppose that 300 first requests from system B and 500 first requests from system C have been received in the first time period. If system A suddenly experiences a traffic surge, sending 2000 first requests to system D, the total first request count is already 800, which is greater than or equal to the second traffic threshold, triggering the second rate limiter. This causes the traffic state to change from unrestricted to restricted, and the number of first requests from system A that can be received normally becomes 0. When the third rate limiters begin to restrict traffic on their respective first channels, since the fourth total count of 300 for system B is greater than or equal to the corresponding third traffic threshold of 300, and the fourth total count of 500 for system C is greater than or equal to the corresponding third traffic threshold of 500, the reception of second requests sent by systems B and C in the first time period stops. Since the third traffic threshold of system A and the difference mentioned above are both 200, the 200 second requests sent by system A can continue to be received in the first time period.

[0121] In one example, suppose that 300 requests from system A and 500 requests from system B have been received in the first time period. The total number of requests received in the first time period reaches 800, which is greater than or equal to the second traffic threshold, triggering the second rate limiter. This causes the traffic status to change from unlimited to limited, and each third rate limiter begins to limit the traffic on its respective first channel. Here, although the third traffic threshold for system C is 500 QPS, the difference between the first and second traffic thresholds is 200. Therefore, after rate limiting, only requests from system C can be received in the first time period if the total number of requests received is less than 200.

[0122] In some embodiments, Figure 3 Step 103 can also be implemented in the following way: when the first channel includes the second channel and the third channel, and the priority of the third channel is lower than that of the second channel, determine the sixth total number of first requests sent by the third channel that have been received in the first time period; when the sixth total number is greater than or equal to the third traffic threshold corresponding to the third channel, stop receiving second requests sent by the third channel in the first time period, and control the seventh total number of second requests sent by the second channel received in the first time period to be less than the difference between the first traffic threshold and the second traffic threshold.

[0123] In practical applications, some callers may have higher priority business operations, such as core payment services compared to ordinary query services; or different callers may have different resource consumption requirements, such as complex calculation requests compared to simple data queries; or some callers may have higher requirements for system stability, such as calls to financial systems compared to ordinary information query requests. Therefore, in scenarios with multiple callers, i.e., multiple primary channels, it is necessary to configure corresponding priorities for each primary channel based on its characteristics. This allows for strict rate limiting policies to be implemented for low-priority callers to prevent excessive system resource consumption; moderate rate limiting policies to be implemented for medium-priority callers to balance system resource allocation; and more lenient rate limiting policies to be implemented for high-priority callers to ensure their business needs are met.

[0124] In practice, if the first channel includes a second and a third channel, and the third channel has a lower priority than the second channel, it is expected that after the traffic status is adjusted to a rate-limited state, requests sent by the second channel will be processed first, while requests sent by the third channel will be rejected or delayed. Therefore, when configuring the third traffic threshold for both the second and third channels, the third traffic threshold for the third channel can be configured to a smaller value, while the third traffic threshold for the second channel can be configured to a larger value. This will strictly limit the number of second requests sent by the third channel after rate limiting. For example, the third traffic threshold for the second channel can be set to 900, and the third traffic threshold for the third channel can be set to 100.

[0125] In actual implementation, if the total number of requests received in the first time period reaches the second traffic threshold, that is, the traffic status is adjusted to the rate limiting status, when rate limiting is implemented, the sixth total number of first requests sent by the third channel received in the first time period can be determined first. The comparison result between the sixth total number and the third traffic threshold corresponding to the third channel can be used to determine whether the number of second requests sent by the third channel can be completely restricted in the first time period, or a small number of second requests sent by the third channel can be received.

[0126] In practice, if the sixth total number is greater than or equal to the third traffic threshold corresponding to the third channel, then the sixth total number of first requests received from the third channel within the first time period has reached the maximum acceptable value under rate-limited conditions. Therefore, when the traffic state is adjusted to rate-limited state, receiving second requests from the third channel within the first time period is directly stopped. For the second channel, the number of second requests received from the second channel within the first time period can be controlled, provided the system is not overloaded. Here, the difference between the first and second traffic thresholds is the maximum number of requests the system can still receive after rate-limiting is triggered. Therefore, the seventh total number of second requests received from the second channel within the first time period needs to be controlled to be less than this difference.

[0127] As an example, assume the first channel includes a second and a third channel. The second channel is system B, and the third channel is system A. System A has a lower priority than system B. For example, system A might be related to B-end business or involve tasks that can be retried, while system B might be related to C-end core business. With the first traffic threshold set to 1000 QPS and the second traffic threshold set to 800 QPS, the third traffic threshold for system B can be set to 900 QPS, and the third traffic threshold for system A can be set to 100 QPS. Then, within the first time period, when the first total number is greater than or equal to the second traffic threshold, causing the traffic status to change from unlimited to limited, if the sixth total number of first requests received from system A is 150, this sixth total number is greater than the third traffic threshold of 100 QPS for system A. Therefore, receiving second requests from system A can be completely stopped within the first time period, and the seventh total number of second requests received from the second channel within the first time period can be controlled to be less than 200. Here, 200 is the difference between the first and second traffic thresholds.

[0128] In some embodiments, if the sixth total number is less than the third traffic threshold corresponding to the third channel, the request quantity of the second and third channels can also be limited in the following way: the eighth total number of second requests received from the third channel within the first time period is controlled to be less than the third traffic threshold corresponding to the third channel, and the sum of the seventh and eighth total numbers is controlled to be less than the difference. In actual implementation, if the sixth total number is less than the third traffic threshold corresponding to the third channel, a small number of second requests sent by the third channel can be received after rate limiting, but it must be ensured that after rate limiting begins, the sum of the seventh total number received from the second channel and the eighth total number received from the third channel is less than the difference between the first and second traffic thresholds.

[0129] Continuing from the previous example, in the first time period, when the first total number is greater than or equal to the second traffic threshold, causing the traffic state to change from unlimited to limited, if the sixth total number of first requests received from system A is 50, and this sixth total number is less than system A's third traffic threshold of 100 QPS, then the eighth total number of second requests received from system A in the first time period can be less than system A's corresponding third traffic threshold, or less than the difference between system A's corresponding third traffic threshold and the sixth total number. Here, it is necessary to ensure that the sum of the seventh and eighth total numbers of second requests received from system B in the first time period is less than the difference between the first and second traffic thresholds, which is 200.

[0130] As an example, if when implementing rate limiting, it is desired that requests from the first channel with lower priority will not be processed (i.e., not received) and will be invoked when the software system load is low, it can be configured as follows: the third traffic threshold of the third channel is 0, and the third traffic threshold of the second channel is equal to the first traffic threshold.

[0131] Thus, assuming the first traffic threshold is 1000 and the second traffic threshold is 800, if the number of first requests received from the second channel within the first time period is 500, then 800-500=300 first requests can still be received from the second channel within the first time period. If the number of first requests received from the second channel within the first time period is 800, then 800-800=0 first requests can still be received from the second channel within the first time period. If the number of first requests received from the second channel within the first time period is 900, then 0 first requests can still be received from the second channel within the first time period. In any of the above cases, when the total number is greater than or equal to the second traffic threshold of 800, causing the traffic status to change from unrestricted to restricted, since the third traffic threshold of the third channel is 0, it will directly stop receiving second requests from the third channel. This means that the lower-priority third channel needs to give way to the higher-priority second channel in terms of system resources.

[0132] By configuring appropriate third traffic thresholds for first channels with different priorities, more refined, flexible, and efficient request quantity control can be achieved for multiple first channels. Targeted setting of third traffic thresholds based on the different priorities of the first channels avoids a "one-size-fits-all" approach to rate limiting. This ensures that requests sent from high-priority second channels are processed first, while requests from low-priority third channels are not processed or are delayed, preventing excessive resource consumption. This improves the stability and resource utilization of the software system, optimizes user experience, reduces system risks, ensures system availability, and supports diverse rate limiting scenarios and needs.

[0133] In some embodiments, Figure 3 Step 103 can also be implemented in the following way: when the second traffic threshold is set to be less than the first value, the ninth total number of second requests sent by the first channel received in the first time period is controlled to be less than the third traffic threshold.

[0134] In practice, the first value can be a value less than or equal to 1, meaning the second traffic threshold is set to be less than 1, for example, directly setting the second traffic threshold to 0. This allows for continuous rate limiting, preventing dynamic switching between unlimited and limited traffic states. Instead, it continuously limits the number of requests to the corresponding first channel through each third rate limiter. In this case, the traffic state is adjusted to rate limiting upon receiving the first request in the first time period. Therefore, the total number of requests sent by the first channel within the first time period is limited to less than the corresponding third traffic threshold.

[0135] As an example, suppose that systems A and B, as the two primary channels, send requests to the called system D. Then, set the first traffic threshold to 1000, the second traffic threshold to 0, the third traffic threshold for system A to 500, and the third traffic threshold for system B to 500. This way, the traffic can be limited to 500 for both system A and system B, and the two primary channels will not interfere with each other.

[0136] In some embodiments, the first traffic threshold, the second traffic threshold, and each third traffic threshold can be set as the maximum number of requests that the called party can receive, so that all first channels can share the total flow control, that is, the principle of first-come, first-served. The entire system will be restricted only when the total number exceeds the first traffic threshold.

[0137] As an example, suppose systems A and B, as the first channels, send requests to the called party, system D. We set the first traffic threshold to 1000, the second traffic threshold to 1000, the third traffic threshold for system A to 1000, and the third traffic threshold for system B to 1000. This way, we can receive one request at a time until the total number of requests exceeds 1000, at which point we will stop receiving all requests.

[0138] In some embodiments, assuming that a certain first channel has a very high priority, even if the callee is overloaded, the number of requests to that first channel will not be limited. In this case, the first traffic threshold can be set to positive infinity, the second traffic threshold can be a suitable value, such as 1000, the third traffic threshold of that first channel is set to positive infinity, and the third traffic threshold of other first channels is set to 0. In this way, when receiving requests, even if the total number of requests exceeds the maximum number of requests that the callee can receive, the entire system will not be limited, and the second requests sent by the highest priority first channel can be continuously received.

[0139] In this manner, when a first request is received from the first channel within the first time period, if the traffic status within the first time period is unrestricted, the total number of requests received within the first time period can be compared with a first traffic threshold and a second traffic threshold. If the total number is less than the first traffic threshold but greater than or equal to the second traffic threshold, the traffic status is triggered to change from unrestricted to restricted. Furthermore, the number of second requests sent by the first channel within the first time period is limited based on the third traffic threshold corresponding to each first channel. This method, through the dual threshold judgment of the first and second traffic thresholds, uses the second traffic threshold as the trigger limit. The system uses a flow status warning value. When the current flow is close to but has not yet reached the first flow threshold (i.e., the first total number is greater than or equal to the second flow threshold, but less than the first flow threshold), it initiates flow limiting in advance to prevent system overload. Furthermore, the system can flexibly decide whether to initiate flow limiting based on the setting of the second flow threshold. In flow limiting mode, the number of requests is limited based on the threshold corresponding to the first channel, enabling flexible configuration of flow control for the first channel and achieving finer-grained flow control to ensure high system availability. In addition, the second flow threshold provides a certain buffer space for the system, ensuring system stability.

[0140] In some embodiments, the server may manage traffic status by: determining a first moment when the traffic status is adjusted from an unlimited state to a limited state, and setting the first moment as the start moment when the traffic status is limited; after limiting the number of second requests sent by a first channel received within a first time period, in response to receiving a second request sent by a first channel within the first time period, determining a second moment when the second request is received, and updating the start moment to the second moment.

[0141] In actual implementation, the traffic status dynamically switches between two states: unlimited and limited. After the traffic status is adjusted to the limited state, the overall request traffic of the software system gradually returns to normal due to the limited processing of each first channel by the third limiter. Therefore, by recording the start time when the traffic status is triggered to the limited state and setting a preset duration, the traffic status can be automatically restored from the limited state to the unlimited state after the preset duration is reached after the start time. Therefore, throughout the process of the limited state, it is necessary to continuously determine the start time of the limited state.

[0142] In actual implementation, when the first total quantity first exceeds or equals the second traffic threshold within the first time period, the traffic status is first triggered to change from an unlimited state to a limited state. Therefore, the first moment when the traffic status is first triggered to change from an unlimited state to a limited state is recorded, and this first moment is taken as the start time of the traffic status becoming limited. Since the traffic status adjustment is triggered every time the first total quantity is determined to be greater than or equal to the second traffic threshold, within the first time period, starting from when the first total quantity exceeds or equals the second traffic threshold, each time a second request is received, a traffic status update is triggered (e.g., determining to remain in a limited state). After limiting traffic, in response to each received second request, it is necessary to determine the second moment when the second request is received and update the second moment to the start time of the traffic status becoming limited.

[0143] In some embodiments, the server may also limit the number of requests in the second time period based on traffic status by: maintaining the rate-limiting state into the second time period, which is after the first time period; and controlling the second total number of third requests received in the second time period to be less than a third traffic threshold in response to the first channel sending a third request in the second time period.

[0144] In actual implementation, during the first time period, after the first total number is greater than or equal to the second traffic threshold, the traffic status is adjusted to a flow-limited state, and the flow-limiting process will last for a period of time. Therefore, when the flow status is maintained in the flow-limited state until the second time period (a time window after the first time period), since the second time period is a new time window, the requests in the second time period will be recounted. Therefore, it is not necessary to adopt the flow-limiting method for the first time period in the above embodiment. Instead, the number of third requests sent by each first channel can be directly limited by each third flow limiter based on the corresponding third traffic threshold, thus limiting the second total number of third requests received in the second time period to be less than the third traffic threshold.

[0145] In some embodiments, the server may adjust the traffic state in the following ways: if the third total number of requests received in the second time period is less than the second traffic threshold, determine the third moment when the third request is received; if the interval between the third moment and the start moment is longer than a preset duration, adjust the traffic state from the rate-limited state to the unrate-limited state, and cancel the control that the second total number is less than the third traffic threshold.

[0146] In actual implementation, due to the flow restriction effect of the third flow limiter on the first channel, the overall number of requests in the system will gradually decrease and tend to normal. Therefore, when the flow status is in the flow-limited state and is maintained until the second time period, the third total number of requests received in the second time period can be continuously detected. If the third total number is less than the second flow threshold, it can be determined whether the flow status can be adjusted from flow-limited to unlimited state.

[0147] Here, the determination of whether the traffic status can be adjusted from the rate-limited state back to the unrestricted state is based on whether the duration of the traffic status in the rate-limited state exceeds a preset duration. Therefore, when detecting whether the third total number exceeds the second traffic threshold, if it is determined that the third total number is less than the third traffic threshold, the third moment when the latest third request is received can be determined, and the interval between the third moment and the start moment can be calculated. When the interval is longer than the preset duration, the traffic status is automatically adjusted from the rate-limited state to the unrestricted state, and the restriction on the number of requests to the first channel is canceled, that is, the restriction that the second total number is less than the third traffic threshold is canceled, so that the system resumes the normal request receiving process. Then, when the total number of requests received in a certain time window is detected to exceed the second traffic threshold again, the number of requests is restricted again by the rate-limiting method in the above embodiment.

[0148] By setting traffic status and dynamically adjusting its switching and recovery in the above manner, the system can respond to traffic changes more intelligently, flexibly, and efficiently. This approach not only protects system stability and avoids excessive rate limiting, but also improves user experience, reduces operational costs, and enhances the system's resilience and adaptability.

[0149] In one specific embodiment Figure 5 This is a schematic diagram of the second flow of the request processing method provided in the embodiments of this application. See also: Figure 5 The request processing method may include the following procedures:

[0150] S401, request to receive.

[0151] During the first time period, begin receiving requests and count the first total number of requests received.

[0152] S402, requesting the first comparison result.

[0153] The first total quantity is compared with the first flow threshold to determine the first comparison result.

[0154] S403, requesting the second comparison result.

[0155] The first total quantity is compared with the second flow threshold to determine the second comparison result.

[0156] S404, requesting the third comparison result.

[0157] For each first channel, determine whether the total number of requests received from the first channel is greater than the corresponding third traffic threshold to obtain the third comparison result.

[0158] S405, determine the traffic status.

[0159] S406, when the flow status is in the flow-limiting state, determines whether to trigger the third flow limiter.

[0160] Based on the third comparison result, it is determined whether the total number of requests received from the first channel exceeds the third rate limiter.

[0161] S407, rate limiting.

[0162] If the third comparison result indicates that the total number of requests received from the first channel exceeds the third rate limiter, then the number of requests from the first channel will be limited.

[0163] S408 determines whether to adjust the flow status.

[0164] If the third comparison result indicates that the total number of requests received from the first channel does not exceed the third rate limiter, then it is determined whether the flow state needs to be restored to the unlimited state.

[0165] If no adjustment of the flow status is required, jump to S410.

[0166] S409 restores the traffic status to the unrestricted state.

[0167] S410, no bandwidth limit.

[0168] S411, when the flow status is unlimited, determine whether to trigger the first flow limiter.

[0169] If the first comparison result indicates that the first total quantity is greater than or equal to the first flow threshold, then jump to S407 to perform overall flow restriction on the system.

[0170] S412, if the first current limiter is not triggered, determine whether the second current limiter should be triggered.

[0171] If the second comparison result indicates that the second current limiter has not been triggered, then jump to S410.

[0172] S413, adjust the traffic status to traffic limiting status.

[0173] If the second comparison result indicates that the second flow limiter is triggered, the flow status is adjusted to the flow limiting status.

[0174] S414 determines whether the third current limiter is triggered.

[0175] If the third comparison result indicates that the third rate limiter is triggered, then the number of requests to the first channel will be limited; if the third comparison result indicates that the third rate limiter is not triggered, then the number of requests to the first channel will not be limited.

[0176] The following description continues to illustrate the exemplary structure of the request processing device 455 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the request processing device 455 of the memory 450 may include:

[0177] The request quantity determination module 4551 is used to determine the first total number of requests received in the first time period in response to receiving a first request sent by the first channel in the first time period, provided that the traffic status in the first time period is unlimited.

[0178] The status adjustment module 4552 is used to adjust the flow status from unlimited state to flow-limited state when the first total quantity is less than the first flow threshold and greater than or equal to the second flow threshold.

[0179] The rate limiting module 4553 is used to limit the number of second requests received from the first channel within a first time period based on the third traffic threshold corresponding to the first channel. The sending time of the second request is after the sending time of the first request.

[0180] In some embodiments, the rate limiting module 4553 is further configured to stop receiving the second request sent by the first channel when the first total number is greater than the first traffic threshold.

[0181] In some embodiments, the state adjustment module 4552 is further configured to determine a first moment when the traffic state is adjusted from an unlimited state to a limited state, and to use the first moment as the start moment when the traffic state is limited; in response to receiving a second request sent by a first channel within a first time period, to determine a second moment when the second request is received, and to update the start moment to the second moment.

[0182] In some embodiments, the state adjustment module 4552 is further configured to maintain the flow-limiting state to a second time period, the second time period being after the first time period; the flow-limiting module 4553 is further configured to control the second total number of third requests received during the second time period to be less than a third flow threshold in response to the first channel sending a third request during the second time period.

[0183] In some embodiments, the state adjustment module 4552 is further configured to determine the third moment when the third request is received if the third total number of requests received in the second time period is less than the second traffic threshold; and if the interval between the third moment and the start moment is longer than a preset duration, adjust the traffic state from the flow-limited state to the unflow-limited state, and the flow-limiting module 4553 cancels the control that the second total number is less than the third traffic threshold.

[0184] In some embodiments, the flow limiting module 4553 is further configured to: stop receiving second requests sent by the first channel during the first time period if the fourth total number of first requests received during the first time period is greater than or equal to the third flow threshold; and determine the difference between the first flow threshold and the second flow threshold, as well as the minimum value of the third flow threshold and the difference, if the fourth total number is less than the third flow threshold, and control the fifth total number of second requests received during the first time period to be less than the minimum value.

[0185] In some embodiments, the flow limiting module 4553 is further configured to: determine a sixth total number of first requests sent by the third channel that have been received in a first time period, provided that the first channel includes a second channel and a third channel and the priority of the third channel is lower than that of the second channel; and stop receiving second requests sent by the third channel in the first time period if the sixth total number is greater than or equal to the third traffic threshold corresponding to the third channel, and control the seventh total number of second requests sent by the second channel that have been received in the first time period to be less than the difference between the first traffic threshold and the second traffic threshold.

[0186] In some embodiments, the flow limiting module 4553 is further configured to control the eighth total number of second requests sent by the third channel received during the first time period to be less than the third traffic threshold corresponding to the third channel when the sixth total number is less than the third traffic threshold corresponding to the third channel, and to control the sum of the seventh total number and the eighth total number to be less than the difference.

[0187] In some embodiments, the flow limiting module 4553 is further configured to control the ninth total number of second requests sent by the first channel received within the first time period to be less than the third flow threshold when the second flow threshold is set to be less than the first value.

[0188] This application provides a computer program product including a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the request processing method described above in this application.

[0189] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the request processing method provided in this application. For example, ... Figure 3 The request processing method is shown.

[0190] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0191] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0192] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0193] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0194] In summary, through the embodiments of this application, when a first request sent by a first channel is received within a first time period, if the traffic status within the first time period is unlimited, the total number of requests received within the first time period can be compared with a first traffic threshold and a second traffic threshold. If the total number is less than the first traffic threshold but greater than or equal to the second traffic threshold, the traffic status is triggered to change from unlimited to limited. Furthermore, the number of second requests sent by the first channel within the first time period is limited according to the third traffic threshold corresponding to each first channel. This method, through the dual threshold judgment of the first and second traffic thresholds, sets the second traffic threshold as... To trigger a warning value for rate limiting, rate limiting is initiated when the current traffic is close to but has not yet reached the first traffic threshold (i.e., the first total number is greater than or equal to the second traffic threshold, but less than the first traffic threshold). This prevents the system from becoming overloaded. Furthermore, the system can flexibly decide whether to enable rate limiting based on the setting of the second traffic threshold. When rate limiting is in effect, the number of requests is limited based on the threshold corresponding to the first channel, enabling flexible configuration of traffic control for the first channel. This allows for finer-grained traffic control of the first channel, ensuring high system availability. In addition, the second traffic threshold provides a buffer zone for the system, ensuring system stability.

[0195] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A request processing method, characterized in that, The method includes: In response to receiving a first request from a first channel during a first time period, and if the traffic status during the first time period is unlimited, determine the first total number of requests received during the first time period. If the first total quantity is less than the first flow threshold and greater than or equal to the second flow threshold, the flow status will be adjusted from the unlimited state to the flow-limited state. Based on the third traffic threshold corresponding to the first channel, the number of second requests received during the first time period sent by the first channel is limited, wherein the sending time of the second request is after the sending time of the first request.

2. The method according to claim 1, characterized in that, After determining the first total number of requests received within the first time period, the method further includes: If the first total number exceeds the first traffic threshold, stop receiving the second request sent by the first channel.

3. The method according to claim 1, characterized in that, After adjusting the flow state from the unlimited state to the limited state, the method further includes: The first moment when the flow state is adjusted from the unlimited state to the limited state is determined, and the first moment is taken as the start moment when the flow state is in the limited state; After limiting the number of second requests received from the first channel within the first time period, the method further includes: In response to receiving the second request sent by the first channel within the first time period, a second time when the second request is received is determined, and the start time is updated to the second time.

4. The method according to claim 3, characterized in that, After updating the start time to the second time, the method further includes: The current limiting state is maintained until the second time period, which is after the first time period; In response to the first channel sending a third request during the second time period, the second total number of the received third requests is controlled to be less than the third traffic threshold.

5. The method according to claim 4, characterized in that, After the second total number of the third requests received by the control is less than the third traffic threshold, the method further includes: If the total number of requests received during the second time period is less than the second traffic threshold, a third moment when the third request is received is determined. If the interval between the third time and the start time is longer than a preset time, the flow state is adjusted from the flow-limited state to the unlimited state, and the control that the second total quantity is less than the third flow threshold is canceled.

6. The method according to claim 1, characterized in that, The limitation on the number of second requests received within the first time period based on the third traffic threshold corresponding to the first channel includes: If the fourth total number of the first requests sent by the first channel received within the first time period is greater than or equal to the third traffic threshold, then receiving the second requests sent by the first channel within the first time period shall be stopped. If the fourth total number is less than the third traffic threshold, the difference between the first traffic threshold and the second traffic threshold, and the minimum value between the third traffic threshold and the difference are determined, and the fifth total number of the second requests sent by the first channel received during the first time period is controlled to be less than the minimum value.

7. The method according to claim 1, characterized in that, The limitation on the number of second requests received within the first time period based on the third traffic threshold corresponding to the first channel includes: If the first channel includes a second channel and a third channel, and the priority of the third channel is lower than that of the second channel, determine the sixth total number of first requests sent by the third channel that have been received during the first time period; If the sixth total number is greater than or equal to the third traffic threshold corresponding to the third channel, the reception of the second request sent by the third channel is stopped during the first time period, and the seventh total number of the second requests sent by the second channel received during the first time period is controlled to be less than the difference between the first traffic threshold and the second traffic threshold.

8. The method according to claim 7, characterized in that, The method further includes: If the sixth total number is less than the third traffic threshold corresponding to the third channel, the eighth total number of second requests sent by the third channel received during the first time period is controlled to be less than the third traffic threshold corresponding to the third channel, and the sum of the seventh total number and the eighth total number is controlled to be less than the difference.

9. The method according to claim 1, characterized in that, The limitation on the number of second requests received from the first channel within the first time period includes: When the second traffic threshold is set to be less than the first value, the ninth total number of second requests sent by the first channel received during the first time period is controlled to be less than the third traffic threshold.

10. A request processing apparatus, characterized in that, The device includes: The request quantity determination module is used to determine the first total number of requests received in the first time period in response to receiving a first request sent by a first channel in the first time period, provided that the traffic status in the first time period is unlimited. The status adjustment module is used to adjust the flow status from the unlimited state to the limited state when the first total quantity is less than the first flow threshold and greater than or equal to the second flow threshold. The rate limiting module is used to limit the number of second requests sent by the first channel within the first time period based on the third traffic threshold corresponding to the first channel, wherein the sending time of the second request is after the sending time of the first request.

11. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the request processing method according to any one of claims 1 to 9.

12. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the request processing method according to any one of claims 1 to 9.

13. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the request processing method according to any one of claims 1 to 9.

Images