Request for traffic limiting methods, apparatus, computer equipment, and readable storage media

Abstract

This application relates to a request traffic limiting method, apparatus, computer device, computer-readable storage medium, and computer program product. Upon receiving a microservice request, an overload protection identifier can be queried using the key value obtained by integrating the Uniform Resource Identifier (URI) and request method. If the overload protection identifier is found, it is determined that the microservice request needs to be overload protected, and thus it is added to the request queue without the need for token allocation for processing. Since it is possible to pre-configure which microservice requests corresponding to URIs and request methods need to be overload protected in case of overload, and to overload protect these requests upon receipt, request traffic can be limited based on the actual system load of the microservice system. Therefore, while implementing reasonable traffic limiting, the limitation on system processing capacity is reduced, and important microservice requests can be responded to promptly.

Description

Technical Field

[0001] This application relates to the field of distributed systems technology, and in particular to a request traffic limiting method, apparatus, computer device, computer-readable storage medium, and computer program product. Background Technology

[0002] Traffic limiting is a protection mechanism for microservice systems when faced with sudden and abnormal traffic spikes. Microservice systems have a large user base and many concurrent requests, making them vulnerable to attacks from abnormal traffic. When peak traffic exceeds the server's maximum load, the stability of online services cannot be guaranteed, and a cascading failure effect may even occur, impacting the normal operation of the entire system. Therefore, request traffic limiting methods are typically introduced into microservice systems. In related technologies, a threshold for the number of pending requests is usually set; when the number of pending requests exceeds the threshold, new requests are rejected. This may limit the system's processing capacity. Summary of the Invention

[0003] Therefore, it is necessary to provide a request traffic limiting method, apparatus, computer device, computer-readable storage medium, and computer program product that can reduce the limitation on the processing capacity of microservice systems in response to the above-mentioned technical problems.

[0004] Firstly, this application provides a method for limiting request traffic, including:

[0005] Upon receiving a microservice request, obtain the requested Uniform Resource Identifier and the request method used;

[0006] The integration result of the unified resource identifier and request method is used as the key value. Based on the key value, the overload protection identifier corresponding to the microservice request is queried.

[0007] If the overload protection flag corresponding to the microservice request is found, it is determined that no token needs to be allocated for the microservice request, and the microservice request is added to the request queue; the request queue is used to provide microservice requests for processing.

[0008] In one embodiment, the method further includes:

[0009] If no overload protection flag is found for the microservice request, and a token exists in the token bucket, a token is selected from the token bucket and assigned to the microservice request. The token indicates that the microservice request has the permission to be added to the request queue.

[0010] Add microservice requests to the request queue.

[0011] In one embodiment, the method further includes:

[0012] If no token is available in the token bucket, the microservice request is rejected.

[0013] In one embodiment, the request queue includes a regular queue and a priority queue; adding a microservice request to the request queue includes:

[0014] Based on the key value, query the priority of the corresponding microservice request;

[0015] If the priority of a microservice request is found, the microservice request is added to the priority queue; otherwise, the microservice request is added to the regular queue.

[0016] In one embodiment, the method further includes:

[0017] In response to the overload protection configuration operation, obtain the triple for configuring overload protection and write it to the overload protection configuration file; the triple includes a Uniform Resource Identifier, a request method, and an overload protection identifier. The overload protection configuration file is used to query the corresponding overload protection identifier for microservice requests based on the key value.

[0018] In one embodiment, the method further includes:

[0019] In response to a priority configuration operation, a triple for configuring priorities is obtained and written to the priority configuration file. The triple includes a Uniform Resource Identifier, a request method, and a priority. The priority configuration file is used to query the priority of microservice requests based on key values.

[0020] In one embodiment, there is a configuration relationship between the number of ordinary queues and priority queues when processing microservice requests; the method further includes:

[0021] During the process of retrieving microservice requests from the regular queue and the priority queue for processing, the corresponding number of microservice requests in the priority queue are processed first, and then the corresponding number of microservice requests in the regular queue are processed, according to the quantity ratio configuration.

[0022] Secondly, this application also provides a request flow limiting device, comprising:

[0023] The acquisition module is used to obtain the requested Uniform Resource Identifier and the request method when a microservice request is received.

[0024] The query module is used to use the integration result of the unified resource identifier and request method as the key value, and based on the key value, query the overload protection identifier corresponding to the microservice request.

[0025] The add module is used to determine that no token needs to be allocated for the microservice request when the overload protection flag corresponding to the microservice request is found, and to add the microservice request to the request queue; the request queue is used to provide microservice requests for processing.

[0026] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0027] Upon receiving a microservice request, obtain the requested Uniform Resource Identifier and the request method used;

[0028] The integration result of the unified resource identifier and request method is used as the key value. Based on the key value, the overload protection identifier corresponding to the microservice request is queried.

[0029] If the overload protection flag corresponding to the microservice request is found, it is determined that no token needs to be allocated for the microservice request, and the microservice request is added to the request queue; the request queue is used to provide microservice requests for processing.

[0030] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0031] Upon receiving a microservice request, obtain the requested Uniform Resource Identifier and the request method used;

[0032] The integration result of the unified resource identifier and request method is used as the key value. Based on the key value, the overload protection identifier corresponding to the microservice request is queried.

[0033] If the overload protection flag corresponding to the microservice request is found, it is determined that no token needs to be allocated for the microservice request, and the microservice request is added to the request queue; the request queue is used to provide microservice requests for processing.

[0034] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0035] Upon receiving a microservice request, obtain the requested Uniform Resource Identifier and the request method used;

[0036] The integration result of the unified resource identifier and request method is used as the key value. Based on the key value, the overload protection identifier corresponding to the microservice request is queried.

[0037] If the overload protection flag corresponding to the microservice request is found, it is determined that no token needs to be allocated for the microservice request, and the microservice request is added to the request queue; the request queue is used to provide microservice requests for processing.

[0038] The aforementioned request traffic limiting method, apparatus, computer device, computer-readable storage medium, and computer program product, upon receiving a microservice request, can first query an overload protection identifier using the key value obtained by integrating the Uniform Resource Identifier (URI) and request method. If an overload protection identifier is found, it is determined that the microservice request needs overload protection, and thus it is added to the request queue without the need for token allocation for processing. Since it is possible to pre-configure which microservice requests corresponding to the URI and request method need overload protection in case of overload, and to overload protect these requests upon receipt, request traffic limiting can be implemented based on the actual system load and overload protection requirements of the microservice system. Therefore, while implementing reasonable traffic limiting, it reduces the limitation on system processing capacity and allows for timely responses to important microservice requests. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a diagram illustrating the application environment of a request traffic limiting method in one embodiment.

[0041] Figure 2 This is a flowchart illustrating a method for requesting traffic limiting in one embodiment;

[0042] Figure 3 This is a flowchart illustrating the request for traffic limiting method in another embodiment;

[0043] Figure 4 This is a simplified flowchart of the request traffic limiting method in yet another embodiment;

[0044] Figure 5 This is a schematic diagram of the interaction process between a user and a rate limiting control agent in one embodiment.

[0045] Figure 6 This is a structural block diagram of a request flow limiting device in one embodiment;

[0046] Figure 7This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0048] Currently, methods for introducing request traffic limiting in microservice systems typically involve setting a threshold for the number of pending requests. When the number of pending requests exceeds this threshold, new requests are rejected. However, this approach lacks flexibility and cannot automatically adjust based on actual system load or network conditions, potentially limiting the system's processing capacity.

[0049] Furthermore, currently, microservice systems typically handle both microservice functionality and request rate limiting, leading to multiple functionalities being performed by the same microservice system. This reduces the system's reliability and increases management complexity. Finally, the current approach of deploying a single rate limiting service for each microservice function, responsible for handling the rate limiting logic uniformly, is flawed. This centralized rate limiting model is vulnerable to single-point attacks, potentially rendering many microservices inaccessible.

[0050] To address the aforementioned problems, embodiments of this application provide a request traffic limiting method, which can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or located in the cloud or on another network server. Terminal 102 can send microservice requests to server 104. Upon receiving a microservice request, server 104 can obtain the requested Uniform Resource Identifier (URI) and the request method used. The integrated result of the URI and request method is used as the key value. Based on the key value, the server queries the corresponding overload protection flag for the microservice request. If the corresponding overload protection flag is found, it is determined that no token needs to be allocated for the microservice request, and based on the key value, the microservice request is added to the request queue; the request queue is used to provide microservice requests for processing.

[0051] The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle systems, and projection devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted displays. Head-mounted displays can be virtual reality (VR) devices, augmented reality devices, and smart glasses. The server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0052] In one exemplary embodiment, such as Figure 2 As shown, a request traffic limiting method is provided, which can be applied to... Figure 1 Taking the server in the example, the explanation includes the following steps 202 to 206. Wherein:

[0053] Step 202: Upon receiving a microservice request, obtain the requested Uniform Resource Identifier and the request method used.

[0054] The Uniform Resource Identifier (URI) is a string used to identify the name of an internet resource. The request method can refer to the Hypertext Transfer Protocol (HTTP) request method, including POST, GET, HEAD, PUT, PATCH, OPTIONS, DELETE, CONNECT, and TRACE. Microservice requests can carry both the URI and the request method.

[0055] Step 204: Use the integration result of the unified resource identifier and request method as the key value, and query the overload protection identifier corresponding to the microservice request based on the key value.

[0056] The integration method can be to concatenate the Uniform Resource Identifier (URI) and the request method; this embodiment does not specifically limit this approach. The overload protection identifier corresponding to the microservice request can be queried in the overload protection configuration file. In the overload protection configuration file, key values ​​and overload protection identifiers can be stored in correspondence. By configuring the overload protection configuration file in advance, it is possible to determine in advance which microservice requests corresponding to URIs and request methods need to be overload protected, rather than being rejected during overload.

[0057] Step 206: If the overload protection flag corresponding to the microservice request is found, it is determined that no token needs to be allocated for the microservice request, and the microservice request is added to the request queue; the request queue is used to provide microservice requests for processing.

[0058] If the overload protection flag for the microservice request is found, it means that overload protection has already been configured for the microservice request with the corresponding Uniform Resource Identifier and request method in case of overload. Therefore, it is unnecessary to assign a token to this microservice request; instead, the microservice request can be directly added to the request queue. Subsequent processing of the microservice request can then be performed by retrieving requests from the request queue.

[0059] In the aforementioned request traffic limiting method, upon receiving a microservice request, the overload protection flag can be queried using the key value obtained by integrating the Uniform Resource Identifier (URI) and the request method. If the overload protection flag is found, it is determined that the microservice request needs overload protection, and thus it is added to the request queue without needing to allocate a token for processing. Because it is possible to pre-configure which URIs and request methods correspond to microservice requests requiring overload protection in case of overload, and to overload protect these requests upon receipt, request traffic limiting can be implemented based on the actual system load and overload protection requirements of the microservice system. Therefore, while implementing reasonable rate limiting, it reduces the limitation on system processing capacity, allowing for timely responses to important microservice requests.

[0060] In one exemplary embodiment, the method further includes:

[0061] If no overload protection flag is found for the microservice request, and a token exists in the token bucket, a token is selected from the token bucket and assigned to the microservice request. The token indicates that the microservice request has the permission to be added to the request queue.

[0062] Add microservice requests to the request queue.

[0063] Specifically, if no overload protection identifier is found for the microservice request, it indicates that overload protection has not been pre-configured for the microservice request corresponding to the Uniform Resource Identifier and request method. Therefore, a token needs to be allocated to the microservice request from the token bucket before it can be added to the request queue to await processing. Thus, it can be determined whether there are tokens in the token bucket. If tokens exist, they can be selected from the token bucket, and the number of tokens in the bucket can be reduced simultaneously. The selected token is then allocated to the microservice request, thereby adding the microservice request to the request queue.

[0064] In this embodiment, since a token can be normally allocated to a microservice request even if no overload protection flag is found, the decision to reject the microservice request is made based on whether a token is allocated. This allows for reasonable request traffic limiting based on the actual system load and overload protection requirements of the microservice system.

[0065] In one exemplary embodiment, the method further includes:

[0066] If no token is available in the token bucket, the microservice request is rejected.

[0067] In this embodiment, since a token can be normally allocated to a microservice request even if no overload protection flag is found, the decision to reject the microservice request is made based on whether a token is allocated. This allows for reasonable request traffic limiting based on the actual system load and overload protection requirements of the microservice system.

[0068] In one exemplary embodiment, the request queue includes a regular queue and a priority queue; adding a microservice request to the request queue includes:

[0069] Based on the key value, query the priority of the corresponding microservice request;

[0070] If the priority of a microservice request is found, the microservice request is added to the priority queue; otherwise, the microservice request is added to the regular queue.

[0071] The processing order of microservice requests in the priority queue can be sorted according to their priority; that is, higher-priority microservice requests can be processed first, followed by lower-priority requests. Microservice requests in the regular queue can be processed in the order they arrived. Furthermore, microservice requests in the priority queue can be processed before those in the regular queue.

[0072] The priority of a microservice request can be queried in the priority configuration file. Key values ​​and priorities are stored in the priority configuration file. By configuring the priority configuration file in advance, it's possible to determine which microservice requests corresponding to Uniform Resource Identifiers and request methods require priority settings. For a given microservice request, if a corresponding priority can be found, it is added to the priority queue and marked with the appropriate priority; otherwise, the microservice request is added to the regular queue.

[0073] In this embodiment, when it is determined that overcurrent protection is needed for a microservice request, there is no need to assign a token to the microservice request. The priority of the microservice request can be further queried. If the priority is found, the microservice request can be added to a priority queue; otherwise, it can be added to a regular queue. Because the priority queue can distinguish the importance of different microservice requests, the possibility of server resources being monopolized by low-priority microservice requests is reduced.

[0074] In one exemplary embodiment, the method further includes:

[0075] In response to the overload protection configuration operation, obtain the triple for configuring overload protection and write it to the overload protection configuration file; the triple includes a Uniform Resource Identifier, a request method, and an overload protection identifier. The overload protection configuration file is used to query the corresponding overload protection identifier for microservice requests based on the key value.

[0076] As mentioned earlier, overload protection configuration files can be set in advance. Specifically, users can configure this in the form of triples by adding triples. Each triple can include a Uniform Resource Identifier (URI), a request method, and an overload protection flag, and these three elements can be stored accordingly.

[0077] In this embodiment, since it is possible to pre-configure which microservice requests corresponding to the Uniform Resource Identifier and request method need to be overloaded under overload conditions, and to overload protect them when the corresponding microservice requests are received, request traffic can be limited according to the actual system load and actual overload protection requirements of the microservice system.

[0078] In one exemplary embodiment, the method further includes:

[0079] In response to a priority configuration operation, a triple for configuring priorities is obtained and written to the priority configuration file. The triple includes a Uniform Resource Identifier, a request method, and a priority. The priority configuration file is used to query the priority of microservice requests based on key values.

[0080] As mentioned earlier, priority configuration files can be set in advance. Specifically, users can configure priority in the form of triples by adding triples. Each triple can include a Uniform Resource Identifier (URI), request method, and priority, and these three elements can be stored accordingly.

[0081] In this embodiment, since priorities can be configured for different microservice requests in advance, microservice requests can be added to a priority queue according to their priorities to distinguish the importance of different microservice requests, thereby reducing the possibility that server resources will be filled by low-priority microservice requests.

[0082] In one exemplary embodiment, a ratio is configured between the number of microservice requests processed in the regular queue and the priority queue; the method further includes:

[0083] During the process of retrieving microservice requests from the regular queue and the priority queue for processing, the corresponding number of microservice requests in the priority queue are processed first, and then the corresponding number of microservice requests in the regular queue are processed, according to the quantity ratio configuration.

[0084] Specifically, since microservice requests in the priority queue are generally processed first, followed by those in the regular queue, this can lead to starvation for microservice requests in the regular queue. To mitigate this, the ratio between the two queues can be configured. For example, a 4:1 ratio can be set, meaning the ratio of priority queues to regular queues is 4:1. In practice, microservice requests from four priority queues can be processed first, followed by one request from the regular queue. This reduces the likelihood of starvation for microservice requests in the regular queue.

[0085] In this embodiment, since rate limiting can be implemented, the processing allocation between the two queues can be distributed by configuring the ratio between the number of ordinary queues and priority queues, thereby reducing the starvation phenomenon caused by microservice requests not being processed in the ordinary queue.

[0086] To facilitate understanding, the complete method flow provided in the embodiments of this application is now described. In these embodiments, the microservice requests mentioned can be initiated by users. Users can initiate microservice requests to the microservice system, and the microservice system can allow users to create multiple identity accounts to initiate microservice requests. The request traffic limiting method provided in these embodiments can be executed by a proxy agent on the server for rate limiting control. The proxy agent can be deployed in the same Pod container as the microservice, so that the microservice system only executes the microservice function, while the rate limiting service is executed by the proxy agent. Based on the above description, see [link to relevant documentation]. Figure 3 The specific implementation can be shown in the following steps:

[0087] Step 302: Create the overload protection configuration file and the priority configuration file.

[0088] Specifically, users can configure an overload protection configuration file for request rate limiting, which can be represented by triples. The triple includes a Uniform Resource Identifier (URI), a request method, and an overload protection flag. The request method can refer to HTTP request methods, including GET, POST, PUT, and Delete, etc. The overload protection flag indicates whether overload protection is required. These triples can be written line by line into the overload protection configuration file.

[0089] Additionally, users can configure priority configuration files, represented by triples. These triples include a Uniform Resource Identifier (URI), a request method, and a priority. Priority characterizes the importance of microservice requests; higher priority means the corresponding microservice request is more likely to be processed. After users create or configure overload protection and priority configuration files, the rate limiting agent can promptly identify file changes and distribute the latest configuration files to the corresponding rate limiting agent for each microservice for updates.

[0090] Step 304: Perform protection and interception processing on microservice requests according to the overload protection configuration file.

[0091] Specifically, when the rate limiting agent receives a microservice request, it first parses the request to obtain the Uniform Resource Identifier (URI) and request method. Next, it reads the overload protection configuration file and converts the triples within it into a hash table. The key value in the hash table is a string obtained by concatenating the URI and request method. Concatenating the URI and request method of the microservice request yields a query key, which is then used to search the hash table. If no corresponding overload protection identifier is found, it means that the microservice request does not require protection or interception. At this point, the normal token verification and rate limiting process can begin.

[0092] If a corresponding overload protection identifier is found, the priority corresponding to that key value can be further queried in the priority configuration file. If a priority exists, the microservice request can be added to the priority queue. Otherwise, it can be added to the regular queue. Because overload protection is implemented, regardless of whether a priority exists, token bucket rate limiting is not required, and requests can be directly added to the queue. Therefore, even if the system enters an overload state, microservice requests that need to be intercepted and protected will not be dropped due to token bucket rate limiting, thus ensuring the processing stability of the core API.

[0093] Step 306: If the microservice request does not require protection and interception, then perform verification token rate limiting.

[0094] Specifically, the rate-limiting agent first checks if there are enough tokens in the token bucket. If there are tokens, it removes one and reduces the current token count. If there are no tokens, rate-limiting protection is applied, and the microservice request is rejected. Once a token is successfully retrieved, the agent parses the microservice request, constructs a key value (a concatenation of a Uniform Resource Identifier and the request method) as an index, and checks the priority configuration file for a priority configuration for the API (the API sending the microservice request). If a priority configuration exists, the microservice request is bound to the priority and added to the priority queue. Otherwise, the microservice request is added to the regular queue. Because it's a priority queue, even later-arriving microservice requests will receive priority processing due to their higher priority.

[0095] Step 308: Divide and process microservice requests in the ordinary queue and the priority queue.

[0096] Specifically, by using dual queues for traffic distribution, rate limiting is implemented while preventing a surge of requests after rate limiting from overwhelming the system. Simultaneously, to ensure timely processing of microservice requests sent through the core API, when a microservice system detects a request in the priority queue, it will first retrieve and process the request from the priority queue. This may cause starvation for regular requests. Therefore, a configurable ratio can be established between priority and regular requests, allowing the microservice system to process a certain number of priority requests before retrieving a regular request from the regular queue for business processing.

[0097] A simplified flowchart of the above process can be found here. Figure 4 Additionally, the interaction between users and the rate limiting agent, as well as the specific rate limiting process, can be found in [reference needed]. Figure 5 The above process proposes a service overload protection strategy based on an overload protection configuration file. This strategy ensures that microservice requests sent through core APIs can still be processed promptly even when the microservice system is overloaded and rate-limited, thus guaranteeing its stability. Furthermore, a priority queue is used to differentiate microservice requests sent through APIs of varying importance, preventing server resources from being overwhelmed by low-priority requests. Finally, a proxy agent is used for rate limiting, rather than the microservices themselves, enabling non-intrusive rate limiting and avoiding the single point of failure inherent in independently deployed rate limiting services.

[0098] The aforementioned request traffic limiting method, based on priority queues and overload protection strategies, can ensure the stability and timeliness of core interfaces and services in microservice systems under high concurrency, as well as traffic limiting operations for non-core APIs. This method can be applied to scenarios including:

[0099] (1) Cloud service and API management platform: In cloud platforms or internal microservice architectures that provide API services, the call frequency and importance of different APIs may vary greatly. This solution can automatically adjust the access permissions and priorities of each API according to the configuration, ensuring the stable operation of critical services, while reasonably allocating resources to other APIs.

[0100] (2) High-performance Web server: For high-traffic websites or applications, especially e-commerce, finance or social platforms, this solution can help prioritize critical operations such as payment and login during peak hours to ensure user experience. At the same time, it limits low-priority activity traffic to prevent system overload.

[0101] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0102] Based on the same inventive concept, this application also provides a request traffic limiting device for implementing the request traffic limiting method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more request traffic limiting device embodiments provided below can be found in the limitations of the request traffic limiting method described above, and will not be repeated here.

[0103] In one exemplary embodiment, such as Figure 6 As shown, a request traffic limiting device is provided, including: an acquisition module 602, a query module 604, and an addition module 606, wherein:

[0104] The acquisition module 602 is used to acquire the requested Uniform Resource Identifier and the request method used when a microservice request is received;

[0105] The query module 604 is used to use the integration result of the unified resource identifier and request method as the key value, and query the overload protection identifier corresponding to the microservice request based on the key value;

[0106] Add module 606, which, upon finding the overload protection flag corresponding to the microservice request, determines that no token needs to be allocated for the microservice request and adds the microservice request to the request queue; the request queue is used to provide microservice requests for processing.

[0107] In an exemplary embodiment, the adding module 606 is further configured to, if no overload protection identifier corresponding to the microservice request is found, select a token from the token bucket if a token exists in the token bucket, and assign it to the microservice request, wherein the token indicates that the microservice request has the permission to be added to the request queue; and add the microservice request to the request queue.

[0108] In one exemplary embodiment, the addition module 606 is further configured to reject the microservice request if no token is present in the token bucket.

[0109] In an exemplary embodiment, the request queue includes a regular queue and a priority queue; the adding module 606 is specifically used to query the priority of a microservice request based on a key value; if the priority of the microservice request is found, the microservice request is added to the priority queue; if the priority of the microservice request is not found, the microservice request is added to the regular queue.

[0110] In an exemplary embodiment, the acquisition module 602 is further configured to, in response to the overload protection configuration operation, acquire a triple for configuring overload protection and write it to the overload protection configuration file; the triple includes a Uniform Resource Identifier, a request method, and an overload protection identifier, and the overload protection configuration file is used to query the corresponding overload protection identifier of a microservice request based on a key value.

[0111] In an exemplary embodiment, the acquisition module 602 is further configured to, in response to the priority configuration operation, acquire a triple for configuring priority and write it to the priority configuration file; the triple includes a Uniform Resource Identifier, a request method, and a priority, and the priority configuration file is used to query the priority of a microservice request based on the key value.

[0112] In an exemplary embodiment, there is a quantity ratio configuration relationship between the ordinary queue and the priority queue when processing microservice requests; the acquisition module 602 is further configured to, in the process of retrieving microservice requests from the ordinary queue and the priority queue for processing, prioritize processing a corresponding number of microservice requests in the priority queue and then process a corresponding number of microservice requests in the ordinary queue according to the quantity ratio configuration relationship.

[0113] Each module in the aforementioned request flow limiting device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can invoke and execute the operations corresponding to each module.

[0114] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores microservice requests. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a request flow limiting method.

[0115] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0116] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0117] Upon receiving a microservice request, obtain the requested Uniform Resource Identifier and the request method used;

[0118] The integration result of the unified resource identifier and request method is used as the key value. Based on the key value, the overload protection identifier corresponding to the microservice request is queried.

[0119] If the overload protection flag corresponding to the microservice request is found, it is determined that no token needs to be allocated for the microservice request, and the microservice request is added to the request queue; the request queue is used to provide microservice requests for processing.

[0120] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0121] If no overload protection flag is found for the microservice request, and a token exists in the token bucket, a token is selected from the token bucket and assigned to the microservice request. The token indicates that the microservice request has the permission to be added to the request queue.

[0122] Add microservice requests to the request queue.

[0123] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0124] If no token is available in the token bucket, the microservice request is rejected.

[0125] In one embodiment, the request queue includes a regular queue and a priority queue; the processor also performs the following steps when executing the computer program:

[0126] Based on the key value, query the priority of the corresponding microservice request;

[0127] If the priority of a microservice request is found, the microservice request is added to the priority queue; otherwise, the microservice request is added to the regular queue.

[0128] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0129] In response to the overload protection configuration operation, obtain the triple for configuring overload protection and write it to the overload protection configuration file; the triple includes a Uniform Resource Identifier, a request method, and an overload protection identifier. The overload protection configuration file is used to query the corresponding overload protection identifier for microservice requests based on the key value.

[0130] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0131] In response to a priority configuration operation, a triple for configuring priorities is obtained and written to the priority configuration file. The triple includes a Uniform Resource Identifier, a request method, and a priority. The priority configuration file is used to query the priority of microservice requests based on key values.

[0132] In one embodiment, there is a proportional configuration relationship between the ordinary queue and the priority queue for handling microservice requests; the processor also performs the following steps when executing the computer program:

[0133] During the process of retrieving microservice requests from the regular queue and the priority queue for processing, the corresponding number of microservice requests in the priority queue are processed first, and then the corresponding number of microservice requests in the regular queue are processed, according to the quantity ratio configuration.

[0134] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0135] Upon receiving a microservice request, obtain the requested Uniform Resource Identifier and the request method used;

[0136] The integration result of the unified resource identifier and request method is used as the key value. Based on the key value, the overload protection identifier corresponding to the microservice request is queried.

[0137] If the overload protection flag corresponding to the microservice request is found, it is determined that no token needs to be allocated for the microservice request, and the microservice request is added to the request queue; the request queue is used to provide microservice requests for processing.

[0138] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0139] If no overload protection flag is found for the microservice request, and a token exists in the token bucket, a token is selected from the token bucket and assigned to the microservice request. The token indicates that the microservice request has the permission to be added to the request queue.

[0140] Add microservice requests to the request queue.

[0141] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0142] If no token is available in the token bucket, the microservice request is rejected.

[0143] In one embodiment, the request queue includes a regular queue and a priority queue; the computer program, when executed by a processor, also performs the following steps:

[0144] Based on the key value, query the priority of the corresponding microservice request;

[0145] If the priority of a microservice request is found, the microservice request is added to the priority queue; otherwise, the microservice request is added to the regular queue.

[0146] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0147] In response to the overload protection configuration operation, obtain the triple for configuring overload protection and write it to the overload protection configuration file; the triple includes a Uniform Resource Identifier, a request method, and an overload protection identifier. The overload protection configuration file is used to query the corresponding overload protection identifier for microservice requests based on the key value.

[0148] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0149] In response to a priority configuration operation, a triple for configuring priorities is obtained and written to the priority configuration file. The triple includes a Uniform Resource Identifier, a request method, and a priority. The priority configuration file is used to query the priority of microservice requests based on key values.

[0150] In one embodiment, there is a proportional configuration relationship between the number of ordinary queues and priority queues for handling microservice requests; when the computer program is executed by the processor, it also performs the following steps:

[0151] During the process of retrieving microservice requests from the regular queue and the priority queue for processing, the corresponding number of microservice requests in the priority queue are processed first, and then the corresponding number of microservice requests in the regular queue are processed, according to the quantity ratio configuration.

[0152] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0153] Upon receiving a microservice request, obtain the requested Uniform Resource Identifier and the request method used;

[0154] The integration result of the unified resource identifier and request method is used as the key value. Based on the key value, the overload protection identifier corresponding to the microservice request is queried.

[0155] If the overload protection flag corresponding to the microservice request is found, it is determined that no token needs to be allocated for the microservice request, and the microservice request is added to the request queue; the request queue is used to provide microservice requests for processing.

[0156] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0157] If no overload protection flag is found for the microservice request, and a token exists in the token bucket, a token is selected from the token bucket and assigned to the microservice request. The token indicates that the microservice request has the permission to be added to the request queue.

[0158] Add microservice requests to the request queue.

[0159] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0160] If no token is available in the token bucket, the microservice request is rejected.

[0161] In one embodiment, the request queue includes a regular queue and a priority queue; the computer program, when executed by a processor, also performs the following steps:

[0162] Based on the key value, query the priority of the corresponding microservice request;

[0163] If the priority of a microservice request is found, the microservice request is added to the priority queue; otherwise, the microservice request is added to the regular queue.

[0164] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0165] In response to the overload protection configuration operation, obtain the triple for configuring overload protection and write it to the overload protection configuration file; the triple includes a Uniform Resource Identifier, a request method, and an overload protection identifier. The overload protection configuration file is used to query the corresponding overload protection identifier for microservice requests based on the key value.

[0166] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0167] In response to a priority configuration operation, a triple for configuring priorities is obtained and written to the priority configuration file. The triple includes a Uniform Resource Identifier, a request method, and a priority. The priority configuration file is used to query the priority of microservice requests based on key values.

[0168] In one embodiment, there is a proportional configuration relationship between the number of ordinary queues and priority queues for handling microservice requests; when the computer program is executed by the processor, it also performs the following steps:

[0169] During the process of retrieving microservice requests from the regular queue and the priority queue for processing, the corresponding number of microservice requests in the priority queue are processed first, and then the corresponding number of microservice requests in the regular queue are processed, according to the quantity ratio configuration.

[0170] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0171] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0172] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for limiting request traffic, characterized in that, The method is executed by a proxy agent on the server used for rate limiting control, and the proxy agent is deployed in the same Pod container as the microservice; The microservice system executes only the microservice functions, while the agent performs rate limiting services; the method includes: Upon receiving a microservice request, obtain the requested Uniform Resource Identifier and the request method used; The Uniform Resource Identifier and the request method are concatenated to form a combined string; the combined string is hashed to generate a fixed-length hash value as a key value; based on the key value, the overload protection identifier corresponding to the microservice request is queried. If the overload protection flag corresponding to the microservice request is found, it is determined that no token needs to be allocated to the microservice request, and the microservice request is added to the request queue. If no overload protection flag is found for the microservice request, the agent determines whether there are enough tokens in the token bucket. If there are tokens in the token bucket, a token is selected from the token bucket and allocated to the microservice request. The token is related to indicating that the microservice request has the permission to be added to the request queue; the microservice request is then added to the request queue. The request queue includes a regular queue and a priority queue, and the microservice requests are distributed and processed through the regular queue and the priority queue; wherein, there is a configuration relationship between the number of microservice requests processed in the regular queue and the priority queue. Adding the microservice request to the request queue includes: querying the priority of the microservice request based on the key value; if the priority of the microservice request is found, adding the microservice request to the priority queue; if the priority of the microservice request is not found, adding the microservice request to the normal queue.

2. The method according to claim 1, characterized in that, The Uniform Resource Identifier (URI) refers to a string used to identify the name of a certain Internet resource, and the request method refers to the Hypertext Transfer Protocol (HTTP) request method, which includes POST, GET, HEAD, PUT, PATCH, OPTIONS, DELETE, CONNECT, and TRACE.

3. The method according to claim 1, characterized in that, The process of routing microservice requests through the ordinary queue and the priority queue includes: During the process of retrieving microservice requests from the regular queue and the priority queue for processing, according to the quantity ratio configuration relationship, a corresponding number of microservice requests in the priority queue are processed first, and then a corresponding number of microservice requests in the regular queue are processed.

4. The method according to claim 1, characterized in that, The method further includes: If no overload protection flag is found for the microservice request, the agent determines whether there are enough tokens in the token bucket. If there are no tokens in the token bucket, the microservice request is rejected.

5. The method according to claim 1, characterized in that, The method further includes: In response to the overload protection configuration operation, a triple for configuring overload protection is obtained and written to the overload protection configuration file; the triple includes a Uniform Resource Identifier, a request method, and an overload protection identifier, and the overload protection configuration file is used to query the corresponding overload protection identifier of a microservice request based on the key value.

6. The method according to claim 1, characterized in that, The method further includes: In response to the priority configuration operation, a triplet for configuring priorities is obtained and written to the priority configuration file; the triplet includes a Uniform Resource Identifier, a request method, and a priority, and the priority configuration file is used to query the priority of microservice requests based on key values.

7. The method according to claim 1, characterized in that, The application scenarios for this method include cloud services and API management platforms, or high-performance web servers.

8. A request flow limiting device, characterized in that, The device includes: The acquisition module is used to obtain the requested Uniform Resource Identifier and the request method when a microservice request is received. The query module is used to concatenate the Uniform Resource Identifier and the request method into a combined string; perform hash calculation on the combined string to generate a fixed-length hash value as a key value; and query the overload protection identifier corresponding to the microservice request based on the key value. The add module is used to determine, upon finding the overload protection identifier corresponding to the microservice request, that no token needs to be allocated to the microservice request, and to add the microservice request to the request queue; the request queue includes a normal queue and a priority queue; The addition module is also used to determine whether there are enough tokens in the token bucket when no overload protection flag corresponding to the microservice request is found. If there are tokens in the token bucket, a token is selected from the token bucket and allocated to the microservice request. The token indicates that the microservice request has the permission to be added to the request queue; the microservice request is then added to the request queue. The module is also used to distribute microservice requests through the ordinary queue and the priority queue; wherein, there is a configuration relationship between the number of microservice requests processed by the ordinary queue and the priority queue. The addition module is also used to add the microservice request to the request queue, including: querying the priority of the microservice request based on the key value; if the priority of the microservice request is found, adding the microservice request to the priority queue; if the priority of the microservice request is not found, adding the microservice request to the ordinary queue.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images