A method and apparatus for instance rate limiting based on shared memory

Abstract

This invention relates to the field of wireless communication technology, specifically to a shared memory-based instance rate limiting method and apparatus. The method is executed by an instance specification rate limiting module and includes: upon receiving a target request, identifying the target listener accessed by the target request through its destination IP address and destination port number; obtaining the target instance ID to which the target listener belongs from the configuration items of the instance specification rate limiting module; searching for a target node in the shared memory area with the target instance ID as the key, and rate limiting the target request based on the instance rate limiting information in the target node; the shared memory area is used to store each instance ID and the instance rate limiting information corresponding to each instance ID. This invention directly limits the rate based on the user instance, unaffected by changes in the number of listeners, thus improving the accuracy and efficiency of rate limiting.

Description

Technical Field

[0001] This invention relates to the field of wireless communication technology, and specifically to an instance rate limiting method and apparatus based on shared memory. Background Technology

[0002] When using load balancing services provided by cloud service providers, users typically purchase load balancing instances of the appropriate specifications through the provider's console. After purchase, the frontend sends the user's configuration information to the control plane, which then sends this configuration information to the Agent. The Agent is an application used to distribute configurations; it processes the received configuration information and writes it to the nginx.conf configuration file (a high-performance HTTP and reverse proxy server). Subsequently, nginx reads this configuration file, retrieves the user's configuration information, and provides services to the user according to this configuration.

[0003] Therefore, under the current architecture, nginx, as the primary carrier of the Layer 7 load balancing service, is unaware of user information and instance specifications. This means that nginx cannot directly rate-limit based on the user's instance specifications when providing services. However, cloud service providers need to rate-limit at the instance level to more accurately control user service and billing. Existing solutions for instance-specific rate limiting generally involve summing the rate limits of all listeners under the current instance. In other words, the rate limits of all listeners under an instance are accumulated to determine the instance's rate limit.

[0004] However, instance rate limiting should be applied directly to the instance itself, rather than by accumulating the rate limiting specifications of listeners. Secondly, when users add or remove listeners without changing their purchased instance specifications, allocating rate limiting specifications becomes very difficult. This is because if instance traffic is still limited based on the sum of the listener rate limiting specifications, the rate limiting will be inaccurate or fail to meet the user's needs. Summary of the Invention

[0005] In view of this, the present invention provides an instance rate limiting method and apparatus based on shared memory to solve the problem that existing solutions cannot directly limit the rate based on the user's instance specifications and lack accuracy.

[0006] In a first aspect, the present invention provides an instance rate limiting method based on shared memory. The method is executed by an instance specification rate limiting module, which is configured within an nginx system architecture. The nginx system architecture also includes a shared memory area connected to the instance specification rate limiting module. The method comprises:

[0007] When a target request is received, the target listener accessed by the target request is identified by the destination IP address and destination port number of the target request.

[0008] Obtain the target instance ID to which the target listener belongs from the configuration items of the instance specification rate limiting module;

[0009] Based on the target instance ID, a target node with the target instance ID as the key is searched in the shared memory area, and the target request is rate-limited based on the instance rate-limiting information in the target node; the shared memory area is used to store each instance ID and the instance rate-limiting information corresponding to each instance ID.

[0010] The above-described solution of this invention achieves direct rate limiting for Tengine instances by developing an instance specification rate limiting module. This allows rate limiting to be applied directly to user instances, rather than based on listeners. It monitors user instance information and, through configuration, distributes the instance ID and instance specification to the underlying data plane. Thus, when processing user requests, the corresponding rate limiting specification can be looked up based on the instance ID, and the request is rate-limited accordingly. This ensures that even if the number of listeners changes, the overall rate limiting of instances remains effectively controlled. Because this invention directly rates-limits user instances, rather than specific listeners, once the user confirms the instance specification, the rate limiting will not change regardless of any changes in the number of listeners. This makes rate limiting more accurate and efficient.

[0011] In one optional implementation, the configuration items of the instance specification rate limiting module include a first configuration item and a second configuration item. The first configuration item is used to indicate the instance ID to which each configured listener belongs, and the second configuration item is used to indicate the instance specification information corresponding to each instance ID.

[0012] In one optional implementation, obtaining the target instance ID to which the target listener belongs from the configuration items of the instance specification rate limiting module includes:

[0013] Obtain the target instance ID to which the target listener belongs from the first configuration item of the instance specification rate limiting module;

[0014] Obtain the target instance specification information corresponding to the target instance ID from the second configuration item of the instance specification rate limiting module.

[0015] In one alternative implementation, the shared memory area organizes the instance IDs and the rate limit information of the user instance corresponding to each instance ID using a lookup tree data structure;

[0016] Each node of the search tree corresponds one-to-one with each instance ID, and each node includes a key identifier and a value identifier. The key identifier is used to store the instance ID of the corresponding node, and the value identifier is used to store the instance rate limiting information corresponding to the instance ID. The instance rate limiting information includes historical access time, historical access volume, and instance specification information.

[0017] In one optional implementation, the step of rate-limiting the target request based on instance rate-limiting information in the target node includes:

[0018] If a target node with the target instance id as the key exists in the shared memory area, then the target instance rate limiting information corresponding to the target instance id is obtained from the value bit identifier of the target node;

[0019] Based on the target instance rate limiting information, determine whether the current access volume of the target instance ID has reached the rate limiting limit of the target instance ID;

[0020] If the current access volume of the target instance ID reaches the rate limit of the target instance ID, the target request is rejected and an HTTP status code is returned;

[0021] If the current access volume of the target instance ID has not reached the rate limit of the target instance ID, then update the target instance rate limit information in the value identifier of the target node, and perform rate limit processing on the target request.

[0022] In one optional implementation, determining whether the current access volume of the target instance ID has reached the rate limit limit of the target instance ID based on the target instance rate limit information includes:

[0023] Obtain the remaining unprocessed request count, configured processing rate, and time interval for processing requests corresponding to the target instance ID, and calculate the remaining number of requests for the target instance ID after the target request is processed;

[0024] When the number of remaining requests exceeds the configured processing rate, it is determined that the current access volume of the target instance ID has reached the rate limit of the target instance ID.

[0025] In one optional implementation, the step of rate-limiting the target request based on instance rate-limiting information in the target node includes:

[0026] If there is no target node in the shared memory area with the target instance id as the key, then a new node is created in the shared memory area with the target instance id as the key and the initial instance rate limiting information corresponding to the target instance id as the value, and the target request is rate-limited.

[0027] Secondly, the present invention provides an instance rate limiting device based on shared memory. The device is executed by an instance specification rate limiting module, which is configured within an nginx system architecture. The nginx system architecture also includes a shared memory area connected to the instance specification rate limiting module. The device comprises:

[0028] The target listener confirmation module is used to confirm the target listener accessed by the target request by using the destination IP address and destination port number of the target request when a target request is received.

[0029] The target instance ID acquisition module is used to obtain the target instance ID to which the target listener belongs from the configuration items of the instance specification rate limiting module;

[0030] The rate limiting module is used to search for target nodes with the target instance ID as the key in the shared memory area based on the target instance ID, and to perform rate limiting processing on the target request based on the instance rate limiting information in the target node; the shared memory area is used to store each instance ID and the instance rate limiting information corresponding to each instance ID.

[0031] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform an instance rate limiting method based on shared memory as described in the first aspect or any corresponding embodiment thereof.

[0032] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute a shared memory-based instance rate limiting method according to the first aspect or any corresponding embodiment described above.

[0033] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute a shared memory-based instance rate limiting method according to the first aspect or any corresponding embodiment thereof.

[0034] The technical solution provided by this invention may include the following beneficial effects:

[0035] This invention implements direct rate limiting for Tengine instances by developing an instance specification rate limiting module. It allows rate limiting to be applied directly to user instances, rather than based on listeners. This allows for monitoring user instance information and, through configuration, distributes the instance ID and instance specification to the underlying data plane. Thus, when processing user requests, the corresponding rate limiting specification can be looked up based on the instance ID, and the request is rate-limited accordingly. This ensures that even if the number of listeners changes, the overall rate limiting of instances remains effectively controlled. Because this invention applies rate limiting directly to user instances, rather than specific listeners, once the user confirms the instance specification, changes to the number of listeners will not alter the rate limiting situation. This makes rate limiting more accurate and efficient. Attached Figure Description

[0036] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0037] Figure 1 This is a flowchart of an instance rate limiting method based on shared memory according to an embodiment of the present invention;

[0038] Figure 2 This is a flowchart of another instance rate limiting method based on shared memory according to an embodiment of the present invention;

[0039] Figure 3 This is a schematic diagram of a shared memory area organized into a search tree data structure according to an embodiment of the present invention;

[0040] Figure 4 This is a flowchart of another instance rate limiting method based on shared memory according to an embodiment of the present invention;

[0041] Figure 5 This is a calculation diagram of the optimized leaky bucket algorithm according to an embodiment of the present invention;

[0042] Figure 6 This is a schematic diagram of the optimized leaky bucket algorithm for determining whether the limit has been exceeded according to an embodiment of the present invention;

[0043] Figure 7 This is a structural block diagram of an instance rate limiting device based on shared memory according to an embodiment of the present invention;

[0044] Figure 8 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] First, some key terms used in this invention will be explained:

[0047] Tengine is a high-performance server primarily used in the network service field to provide load balancing services. Its infrastructure is similar to the open-source Nginx architecture, but it offers richer functionality and greater scalability. When handling network requests, Tengine can better allocate resources, improving service efficiency and reliability, and adapting to more complex network scenarios and business needs. For example, in high-concurrency network environments, Tengine can more flexibly handle a large number of user requests, ensuring stable service operation.

[0048] A load balancer instance is a concrete manifestation of a running load balancing service. To use a load balancing service, you must first create a load balancer instance. It is the actual unit that implements load balancing functionality, responsible for distributing network requests appropriately across multiple backend servers to prevent performance issues on a single server due to excessive load, thus ensuring the efficient operation of the entire system.

[0049] Listener: In Layer 7 load balancing, the listener is a crucial component for implementing load balancing services and serves as the entry point for network requests. Its primary implementation involves the operating system listening on a specific IP address and port combination. For example, when a user enters a URL into their browser to access a website, the request first reaches the listener. The listener, based on pre-defined rules, decides whether to forward the request to the target backend server for processing. A load balancing instance requires at least one listener, and multiple listeners can handle different types of requests according to different rules.

[0050] Instance Specifications: For each load balancing instance, the instance specifications are a standard used to measure the extent to which a user utilizes the load balancing service. In Layer 7 load balancing services, QPS (Requests Per Second) is typically used as the metric. Cloud service providers offer a variety of load balancing instance specifications for users to choose from, based on different business needs and user scales.

[0051] Leaky Bucket Algorithm: A commonly used algorithm for rate limiting. Its main purpose is to control the rate at which data is injected into the network. In network services, if there is a sudden surge in user requests (burst traffic), the leaky bucket algorithm can process these requests sequentially at a set rate, preventing the server from crashing due to a sudden high load and ensuring the stability of the service and the normal use of other users.

[0052] The agent is a dedicated application for distributing configurations, analogous to a configuration delivery mechanism for Nginx. Its primary task is to pass relevant configuration information to Nginx. When a user makes configuration changes to the load balancing service in the cloud service provider's console, this configuration information is transmitted through the frontend and control plane to the agent. The agent then organizes these configurations and writes them into the Nginx configuration file, nginx.conf, enabling Nginx to provide the corresponding services based on the new configuration.

[0053] According to an embodiment of the present invention, this embodiment provides an nginx system architecture with an added instance specification rate limiting module. The nginx system architecture also includes a shared memory area connected to the instance specification rate limiting module (i.e., the lb_inst_qps module).

[0054] Furthermore, User A initiates a request through a browser, aiming to access a specific service. User A's request first reaches the load balancer; in this embodiment, there are two load balancer instances, labeled Server A and Server B. The load balancer's function is to distribute requests to the backend Nginx server cluster according to preset rules (such as round-robin, least connections, etc.). The console is the interface for users to configure and manage the load balancing service, where users configure listeners and instance specifications. The control plane of the console is the component responsible for handling and managing configuration information. It receives configuration information from the console and distributes this information to the agent. The agent is a configuration distributor responsible for synchronizing the control plane's configuration information to the Nginx servers. The Nginx servers use the nginx.conf file to store configuration information, which contains information such as the server's IP address, port number, and a list of backend servers. The Nginx server cluster is the group of servers that actually handle user requests. Each Nginx server runs Nginx software and processes requests according to the configuration in the nginx.conf file. The backend servers are the servers that ultimately provide the service; the Nginx server forwards requests to the backend servers according to the configuration.

[0055] Furthermore, this embodiment introduces an instance specification rate limiting module (lb_inst_qps module). That is, by utilizing the flexible third-party extension module architecture provided by the nginx system architecture, the lb_inst_qps module was developed, which enables nginx to directly perceive user instances and instance specifications. Through this instance specification rate limiting module, rate limiting can be applied to each user instance, making up for the current lack of nginx functionality in this area. At the same time, it can be widely used in the load balancing services of cloud network vendors.

[0056] Due to the original design architecture of nginx, there is currently a lack of functionality for instance-based rate limiting, and the native nginx architecture does not focus on the instance parameter. Therefore, this embodiment introduces an instance specification rate limiting module (lb_inst_qps module). This module requires the addition of some configuration items, the most critical of which are the lb_inst_id and lb_inst_qps_rate configuration items. The lb_inst_id configuration item is used to indicate the instance ID to which each configured listener belongs, while the lb_inst_qps_rate configuration item is used to indicate the specification of the instance.

[0057] In the implementation of the instance specification rate limiting module (lb_inst_qps module), this embodiment first allocates a shared memory area. Within this shared memory area, instance specification information and the current access volume are stored in key-value format. The key is the instance ID of each request, and the value is a structure storing information such as the current access volume, access time, and instance specification. Whenever a request arrives, the listener accessed by the request can be identified through the destination IP address and port number. Simultaneously, based on the configuration of the instance specification rate limiting module (lb_inst_qps module), the instance ID to which the listener belongs can be obtained. Subsequently, the specific specification of that instance ID can be queried, and then the optimized leaky bucket algorithm can be used to rate limit the request. This achieves the goal of rate limiting user requests based on instances.

[0058] In summary, this embodiment achieves direct rate limiting for Tengine instances by developing an instance specification rate limiting module. This allows rate limiting to be applied directly to user instances, rather than based on listeners. It monitors user instance information and, through configuration, distributes the instance ID and instance specification to the underlying data plane. Thus, when processing user requests, the corresponding rate limiting specification can be found based on the instance ID, and the request is rate-limited accordingly. This ensures that even if the number of listeners changes, the overall rate limiting of instances remains effectively controlled. Because this embodiment directly rates-limits user instances, rather than specific listeners, once the user confirms the instance specification, the rate limiting will remain unchanged regardless of any changes in the number of listeners. This makes rate limiting more accurate and efficient.

[0059] According to an embodiment of the present invention, an instance rate limiting method based on shared memory is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0060] This embodiment provides a shared memory-based instance rate limiting method. This method is executed by an instance specification rate limiting module, which is set up within the nginx system architecture. The nginx system architecture also includes a shared memory area connected to this instance specification rate limiting module. Figure 1 This is a flowchart of an instance rate limiting method based on shared memory according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0061] Step S101: When a target request is received, the target listener accessed by the target request is confirmed by the destination IP address and destination port number of the target request.

[0062] Furthermore, in the network request processing, this embodiment first determines the target listener corresponding to the target request. Because the listener is the first point of contact after a network request enters the system, identifying the target listener is crucial to obtaining related instance information, thus providing a foundation for subsequent rate-limiting processing. When the system receives a target request, it extracts the destination IP address and destination port number from the request's packet. In the load-balanced Nginx system architecture, each listener is associated with a specific IP address and port number; through this mapping relationship, the target listener accessed by the target request can be identified.

[0063] Step S102: Obtain the target instance ID to which the target listener belongs from the configuration items of the instance specification rate limiting module.

[0064] Furthermore, obtaining the target instance ID to which the target listener belongs is a crucial step in associating the listener with a specific instance. Only by knowing the target instance ID can information such as the instance's rate-limiting rules be obtained, enabling accurate rate-limiting of requests. In this embodiment, the search is performed in the configuration items of the instance specification rate-limiting module (i.e., the lb_inst_qps module). These configuration items were loaded during the previous initialization process and contain the correspondence between listeners and instance IDs. By searching these configuration items using the relevant identifiers or configuration information of the target listener, its target instance ID can be obtained.

[0065] Step S103: Based on the target instance ID, search for the target node with the target instance ID as the key in the shared memory area, and perform rate limiting processing on the target request based on the instance rate limiting information in the target node; the shared memory area is used to store each instance ID and the instance rate limiting information corresponding to each instance ID.

[0066] Furthermore, in this embodiment, based on the obtained target instance ID, the corresponding target node is located in the shared memory area, and then the instance rate-limiting information in the target node is used to rate-limit the target request. The shared memory area stores each instance ID and related instance rate-limiting information. By quickly finding the target node, rate-limiting information can be efficiently obtained and processed to achieve accurate rate-limiting of requests and ensure that the entire system operates stably within the instance specification range.

[0067] This embodiment first searches within a shared memory area, which can store information in the form of a red-black tree combined with a hash table. In the red-black tree, nodes use the instance ID as the key, and the node value is a structure containing instance rate-limiting information (such as access volume, access time, instance specifications, etc.). Once a target node with the target instance ID as the key is found, this embodiment determines whether the target request exceeds the rate limit based on the instance rate-limiting information in the node (combined with the leaky bucket algorithm, etc.). If the request volume does not exceed the rate limit, the request is allowed to continue processing, and the access volume and other information in the node are updated (e.g., the access volume is incremented by 1, the access time is updated, etc.). If the rate limit is exceeded, the request is rejected, and the corresponding HTTP status code is returned.

[0068] In summary, this embodiment achieves direct rate limiting for Tengine instances by developing an instance specification rate limiting module. This allows rate limiting to be applied directly to user instances, rather than based on listeners. It monitors user instance information and, through configuration, distributes the instance ID and instance specification to the underlying data plane. Thus, when processing user requests, the corresponding rate limiting specification can be found based on the instance ID, and the request is rate-limited accordingly. This ensures that even if the number of listeners changes, the overall rate limiting of instances remains effectively controlled. Because this embodiment directly rates-limits user instances, rather than specific listeners, once the user confirms the instance specification, the rate limiting will remain unchanged regardless of any changes in the number of listeners. This makes rate limiting more accurate and efficient.

[0069] This embodiment provides another instance rate limiting method based on shared memory. This method is executed by an instance specification rate limiting module, which is set in the nginx system architecture. The nginx system architecture also has a shared memory area connected to the instance specification rate limiting module. Figure 2 This is a flowchart of another instance rate limiting method based on shared memory according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0070] Step S201: Upon receiving a target request, the target listener accessed by the target request is identified through the destination IP address and destination port number of the request. For details, please refer to [link to relevant documentation]. Figure 2 Step S201 of the illustrated embodiment will not be described again here.

[0071] Step S202: Obtain the target instance ID to which the target listener belongs from the configuration items of the instance specification rate limiting module.

[0072] In one optional implementation, the configuration items of the instance specification rate limiting module include a first configuration item and a second configuration item. The first configuration item is used to indicate the instance ID to which each configured listener belongs, and the second configuration item is used to indicate the instance specification information corresponding to each instance ID.

[0073] In one optional implementation, step S202 includes:

[0074] Obtain the target instance ID to which the target listener belongs from the first configuration item of the instance specification rate limiting module;

[0075] Obtain the target instance specification information corresponding to the target instance ID from the second configuration item of the instance specification rate limiting module.

[0076] Furthermore, the configuration items of the instance specification rate limiting module contain key information. The first configuration item (lb_inst_id configuration item) establishes the association between the listener and its corresponding instance ID, while the second configuration item (lb_inst_qps_rate configuration item) clarifies the instance specification information corresponding to each instance ID. These two configuration items work together to enable the system to accurately obtain the detailed specifications of the instance to which the target listener belongs, providing a basis for subsequent rate limiting operations. In this embodiment, the system first finds the target instance ID from the first configuration item based on the identifier or related configuration content of the target listener. Then, it retrieves the corresponding target instance specification information from the second configuration item using the obtained target instance ID. For example, if the first configuration item records that listener A belongs to instance 1, then the target instance ID is determined to be 1. Next, the system searches for the instance specification information corresponding to instance 1 in the second configuration item, such as the maximum number of requests per second (QPS value).

[0077] Furthermore, this embodiment combines the third-party module extension interface provided by nginx to develop an instance specification rate limiting module (i.e., the lb_inst_qps module). This module is developed based on the nginx third-party extension structure, adding configuration items and processing modules, and intervening in the access phase of nginx request processing. From an implementation perspective, the instance specification rate limiting module (i.e., the lb_inst_qps module) first adds structures such as ngx_http_lb_inst_qps_node_t and ngx_http_lb_inst_qps_limit_t, representing the red-black tree node information used to store rate limiting information, the shared memory block storing the rate limiting information, and the specific rate limiting value, respectively. Subsequently, it adds instructions such as lb_inst_qps_zone (initializing the rate limiting shared memory block), lb_inst_qps_limit (specifying the specific rate limiting value), and lb_inst_qps_status (user-customizable nginx return code when the rate limiting value is reached to distinguish it from other abnormal situations), as well as functions for finding the user instance ID and determining whether the user has reached the rate limiting value.

[0078] In other words, as a commonly used server software, the Nginx system architecture provides third-party module extension interfaces. This embodiment utilizes this feature to develop an instance specification rate limiting module (i.e., the lb_inst_qps module). This allows specific functions to be added to Nginx without changing its core architecture, enhancing Nginx's ability to limit instance rates and meeting the needs of precise rate limiting for load balancing instances in cloud services. This instance specification rate limiting module (i.e., the lb_inst_qps module) intervenes in the access phase of Nginx's request processing. At this phase, the instance specification rate limiting module (i.e., the lb_inst_qps module) can obtain relevant request information, such as the destination IP address and port number, thus providing an opportunity for instance-based rate limiting operations. The gx_http_lb_inst_qps_node_t structure is used to represent the red-black tree node information for subsequently storing rate limiting information. A red-black tree is a self-balancing binary search tree that provides efficient search, insertion, and deletion operations when processing large amounts of data. This structure allows the module to effectively organize and manage instance rate-limiting data within a red-black tree, facilitating quick location and updating of rate-limiting information for specific instances. The `ngx_http_lb_inst_qps_limit_t` structure represents the shared memory block storing rate-limiting information and the specific rate-limiting value. The use of shared memory improves data access speed, reduces data copying overhead, and enables multiple processes or threads to quickly access and modify rate-limiting information. This structure clarifies the format and related information of the rate-limiting value stored in shared memory, ensuring accurate storage and retrieval of the rate-limiting value. The instance-specific rate-limiting module (i.e., the `lb_inst_qps` module) also adds a series of functions, such as a function to find the user's instance ID, used to determine the instance to which the request belongs when processing a request; and a function to determine if a user has reached the rate-limiting value, calculating and comparing the request volume with the rate-limiting value to decide whether to allow the request to continue processing. These functions, working in conjunction with the new structures and instructions, implement a complete process from obtaining request information, finding the instance ID, determining the rate-limiting status, to finally processing the request (accepting or rejecting), ensuring the accurate implementation of instance-based rate-limiting functionality.

[0079] Step S203: Based on the target instance ID, search for the target node in the shared memory area using the target instance ID as the key. This shared memory area is used to store each instance ID and the instance rate-limiting information corresponding to each instance ID.

[0080] In one alternative implementation, the shared memory area organizes the instance IDs and the rate limit information for each user instance ID using a lookup tree data structure.

[0081] Each node in the search tree corresponds one-to-one with each instance ID, and each node includes a key identifier and a value identifier. The key identifier is used to store the instance ID of the corresponding node, and the value identifier is used to store the instance rate limiting information corresponding to the instance ID. The instance rate limiting information includes historical access time, historical access volume, and instance specification information.

[0082] Furthermore, in this embodiment, the shared memory area uses a search tree (such as a red-black tree) data structure to organize each instance ID and its corresponding instance rate-limiting information. This structure facilitates rapid data search and management, improving system performance. Each node in the search tree corresponds to an instance ID. The key identifier stores the instance ID, and the value identifier stores instance rate-limiting information including historical access time, historical access volume, and instance specification information. In this embodiment, based on the target instance ID, a search is performed in the search tree of the shared memory area. Starting from the root node, the tree nodes are traversed step by step according to the comparison between the key identifier and the target instance ID until a target node whose key identifier matches the target instance ID is found. For example, if the target instance ID is 5, starting from the root node of the search tree, the key identifier of the node is compared with the value of 5, and the search continues to the left or right subtree until a target node with a key of 5 is found.

[0083] Furthermore, during initialization, the instance specification rate limiting module (i.e., the lb_inst_qps module) first needs to load the lb_inst_id and lb_inst_qps_rate configuration items from the configuration file, and then allocate a shared memory block for storing instance ID information, the access volume corresponding to each ID, and instance specification information. In the shared memory, user instance information is actually stored in the form of a red-black tree plus a hash table; please refer to [link to relevant documentation]. Figure 3 The diagram illustrates how the shared memory area organizes nodes using a search tree data structure. The lb_inst_qps module maintains a red-black tree, where each node stores information in key-value pairs, where the key is the user instance ID (e.g., ...). Figure 3 (lb_id1 to lb_id5 in the data), value (e.g.) Figure 3 The `qps_node` in the red-black tree is a complex structure containing information such as the last access time of the user instance and the instance's specifications. Initially, this red-black tree is empty. Using a combination of a red-black tree and a hash table significantly improves query efficiency, and even with a large number of instances, search efficiency does not decrease significantly. After completing the above operations, the instance-specific rate limiting module (i.e., the `lb_inst_qps` module) will wait for incoming requests.

[0084] In other words, loading the `lb_inst_id` and `lb_inst_qps_rate` configuration items in the configuration file is a crucial first step during module initialization. The `lb_inst_id` configuration item specifies the instance ID to which each configured listener belongs, forming the basis for subsequently associating listeners with instances. For example, in a complex network architecture, multiple listeners may serve different instances; this configuration item accurately identifies the instance corresponding to each listener. The `lb_inst_qps_rate` configuration item specifies the instance's specifications, typically measured in QPS (queries per second), determining the upper limit of the instance's request handling capacity. Allocating shared memory blocks is for efficient storage of instance-related information. Shared memory allows multiple processes or modules to quickly access the same memory area, avoiding frequent data copying across different memory spaces, thus improving data access speed. In this embodiment, it is necessary to store instance ID information, the access volume corresponding to each ID, and instance specification information, etc. This information needs to be frequently read and updated when processing requests; using shared memory can significantly improve system performance. In shared memory, user instance information is stored using a red-black tree combined with a hash table. Each node in a red-black tree is stored in key-value pairs. The key is the user instance ID, allowing for quick location of the corresponding node using the instance ID. The value is a complex structure containing important information such as the last access time of the user instance and the instance specifications. For example, when a request arrives, the system can quickly find the corresponding node in the red-black tree using the instance ID, and then retrieve information such as the last access time and instance specifications from the node's structure to determine whether rate limiting is necessary and how to implement it.

[0085] Step S204: If there is a target node in the shared memory area with the target instance id as the key, then obtain the target instance rate limiting information corresponding to the target instance id from the value bit identifier of the target node; based on the target instance rate limiting information, determine whether the current access volume of the target instance id has reached the rate limiting limit of the target instance id.

[0086] In one optional implementation, step S304 includes:

[0087] Get the number of remaining unprocessed requests corresponding to the target instance ID, the configured processing rate, and the time interval for processing requests. Calculate the number of remaining requests for the target instance ID after the target request is processed.

[0088] When the number of remaining requests exceeds the configured processing rate, it is determined that the current access volume of the target instance ID has reached the rate limit of the target instance ID.

[0089] Furthermore, if the target node is found in the shared memory area, this embodiment obtains the target instance rate-limiting information from its value identifier, including historical access time, historical access volume, and instance specification information (such as configured processing rate). Then, it calculates the remaining number of requests for the target instance ID after the target request has been processed. The calculation formula is based on an optimized leaky bucket algorithm to determine if the rate-limiting limit has been reached: the calculated remaining number of requests is compared with the configured processing rate. When the remaining number of requests exceeds the configured processing rate, it is determined that the current access volume of the target instance ID has reached the rate-limiting limit. This means that if the request continues to be processed, it may exceed the instance's processing capacity, affecting system stability.

[0090] Furthermore, when a user request arrives, please refer to [link / reference]. Figure 4 The flowchart illustrates another instance rate limiting method based on shared memory. In this embodiment, the destination IP address and destination port number of each request packet can be used to determine the listener that the request wants to access. Subsequently, the instance specification rate limiting module (i.e., the lb_inst_qps module) will look up the instance ID and instance specification of the listener in the configuration. After obtaining the above information, the instance specification rate limiting module (i.e., the lb_inst_qps module) will search for the instance ID in shared memory and check if there is a node with the instance ID as the key in the current red-black tree. If there is, it means that the instance has been accessed, so the value of the node is obtained. Based on the information in the value, the access volume of the instance ID is calculated using the leaky bucket algorithm to see if it has reached the limit. If it has reached the limit, the request is rejected and an HTTP status code is returned; if it has not reached the limit, the request is accepted, the access time in the red-black tree node is updated, the access volume is incremented by 1, and then control is returned to nginx, which continues to process the request. If no node with the instance ID as the key is found in the red-black tree, it means that the instance is being accessed for the first time. In this case, a new red-black tree node is added with the instance ID as the key, recording the current access time. The access count is initially 0, and now the access count is incremented by 1. The node is then inserted into the red-black tree, and control is returned to nginx for further processing of the request.

[0091] In step S205, if the current access volume of the target instance id reaches the rate limit of the target instance id, the target request is rejected and an HTTP status code is returned; if the current access volume of the target instance id does not reach the rate limit of the target instance id, the target instance rate limit information in the value bit identifier of the target node is updated, and the target request is rate-limited.

[0092] Furthermore, if the current access volume of the target instance ID is determined to have reached the rate limit, the system rejects the target request and returns an HTTP status code to inform the client that the request is restricted. This is to protect system resources and prevent overload. If the rate limit has not been reached, the target instance rate limit information in the target node's value identifier is updated, such as updating the historical access time to the current time and incrementing the historical access volume by 1. Then, the target request is rate-limited, meaning the request is forwarded to the backend server for processing according to the normal process, but will be subject to instance specification limitations to ensure that the overall system operates within the specified load range.

[0093] Step S206: If there is no target node in the shared memory area with the target instance id as the key, then in the shared memory area, a corresponding new node is created with the target instance id as the key and the initial instance rate limiting information corresponding to the target instance id as the value, and rate limiting processing is performed on the target request.

[0094] Furthermore, if no target node with the target instance ID as the key is found in the shared memory area, it indicates that the instance is being accessed for the first time. In this case, this embodiment creates a new node in the shared memory area with the target instance ID as the key and the initial instance rate limiting information corresponding to the target instance ID (e.g., initial access volume is 0, initial access time is the current time, and instance specification information is obtained from the configuration items) as the value. This prepares for subsequent request processing and rate limiting statistics for this instance. Afterwards, rate limiting is applied to the target request, the request is forwarded to the backend server, and the access volume and other information of this instance are statistically analyzed and managed.

[0095] Furthermore, this embodiment is based on the leaky bucket algorithm, but compared to the traditional leaky bucket algorithm, this embodiment has made some design differences to meet the current business needs. Firstly, the overall implementation is based on the leaky bucket algorithm; therefore, the main calculation formulas are based on the fundamental principles of the leaky bucket algorithm. The following are the main judgment criteria; please refer to [link / reference]. Figure 5 The diagram illustrates the calculation of the optimized leaky bucket algorithm. Here, `lr` represents the red-black tree node currently hit by `lb_id`, `lr->excess` represents the total number of unprocessed requests remaining in that node, `rate` represents the request processing rate configured in the client's configuration file, `ms` represents the time interval between the last processing and the current processing (in milliseconds), and `+1000` indicates the current request being processed (this value is initially 1, but for calculation convenience, the entire formula is multiplied by 1000, hence 1000 here instead of 1). This formula calculates the number of remaining requests after processing the current request. When this number exceeds the remaining processing capacity, the bucket overflows, and subsequent requests will be rejected because the rate limit has been reached. Please refer to [link to relevant documentation]. Figure 6The diagram illustrates the optimized leaky bucket algorithm's logic for determining if a request exceeds a limit. The condition is that if the remaining number of requests exceeds the allowed rate, the excess requests are rejected. Traditional leaky bucket algorithms allow for sudden traffic spikes, judging by whether the remaining request volume exceeds the allowed spike increment, i.e., the `limit->burst` parameter. However, in the optimized leaky bucket algorithm of this embodiment, access is directly rejected when the user request volume reaches the threshold. Theoretically, if sudden spikes were not allowed, `limit->burst` could be set to 0. However, through deduction and verification, this threshold cannot be 0; it must be `rate`. When the limit is high, greater than 1000 qps (i.e., more than one request must be processed per ms), and the requests arrive in a concentrated time window, if the threshold is set to 0, the processing time difference between two requests will be less than 1ms. According to the formula, the excess value cannot be reduced. Therefore, in this embodiment, the excess value needs to be set to rate, which is equivalent to manually distributing the requests that may arrive in a concentrated manner to each time scale of the unit time window. If the threshold is 0, it is equivalent to only calculating the number of requests within the current surge time window, and the calculated rate limit value is too low. Therefore, the threshold is set to rate, which is actually rate*1000ms / 1000, that is, the total number of requests that can be processed in 1 second (1000ms) (i.e., qps per second of calculation time). When the number of remaining requests to be processed is greater than this value, it means that the bucket is about to overflow, and new requests will be rejected because they have reached the upper limit of the leaky bucket algorithm.

[0096] In summary, this embodiment achieves direct rate limiting for Tengine instances by developing an instance specification rate limiting module. This allows rate limiting to be applied directly to user instances, rather than based on listeners. It monitors user instance information and, through configuration, distributes the instance ID and instance specification to the underlying data plane. Thus, when processing user requests, the corresponding rate limiting specification can be found based on the instance ID, and the request is rate-limited accordingly. This ensures that even if the number of listeners changes, the overall rate limiting of instances remains effectively controlled. Because this embodiment directly rates-limits user instances, rather than specific listeners, once the user confirms the instance specification, the rate limiting will remain unchanged regardless of any changes in the number of listeners. This makes rate limiting more accurate and efficient.

[0097] This embodiment also provides a shared memory-based instance rate limiting device, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0098] This embodiment provides an instance rate limiting device based on shared memory, such as... Figure 7 As shown, it includes:

[0099] The target listener confirmation module 701 is used to confirm the target listener accessed by the target request by using the destination IP address and destination port number of the target request when a target request is received.

[0100] The target instance ID acquisition module 702 is used to obtain the target instance ID to which the target listener belongs from the configuration items of the instance specification rate limiting module;

[0101] The rate limiting module 703 is used to search for a target node with the target instance ID as the key in the shared memory area based on the target instance ID, and to perform rate limiting processing on the target request based on the instance rate limiting information in the target node; the shared memory area is used to store each instance ID and the instance rate limiting information corresponding to each instance ID.

[0102] In some optional implementations, the configuration items of the instance specification rate limiting module include a first configuration item and a second configuration item. The first configuration item is used to indicate the instance ID to which each configured listener belongs, and the second configuration item is used to indicate the instance specification information corresponding to each instance ID.

[0103] In some optional implementations, the target instance ID acquisition module 702 is further configured to:

[0104] Obtain the target instance ID to which the target listener belongs from the first configuration item of the instance specification rate limiting module;

[0105] Obtain the target instance specification information corresponding to the target instance ID from the second configuration item of the instance specification rate limiting module.

[0106] In some alternative implementations, the shared memory area organizes the individual instance IDs and the rate limit information for each user instance ID using a lookup tree data structure;

[0107] Each node in the search tree corresponds one-to-one with each instance ID, and each node includes a key identifier and a value identifier. The key identifier is used to store the instance ID of the corresponding node, and the value identifier is used to store the instance rate limiting information corresponding to the instance ID. The instance rate limiting information includes historical access time, historical access volume, and instance specification information.

[0108] In some alternative implementations, the speed limiting processing module 703 is further configured to:

[0109] If a target node with the target instance ID as the key exists in the shared memory area, then the rate limiting information of the target instance corresponding to the target instance ID is obtained from the value bit identifier of the target node;

[0110] Based on the rate limit information of the target instance, determine whether the current access volume of the target instance ID has reached the rate limit limit of the target instance ID;

[0111] If the current access volume of the target instance ID reaches the rate limit of the target instance ID, the target request will be rejected and an HTTP status code will be returned.

[0112] If the current access volume of the target instance ID has not reached the rate limit limit of the target instance ID, then update the target instance rate limit information in the value bit identifier of the target node, and rate limit the target request.

[0113] In some alternative implementations, the speed limiting processing module 703 is further configured to:

[0114] Get the number of remaining unprocessed requests corresponding to the target instance ID, the configured processing rate, and the time interval for processing requests. Calculate the number of remaining requests for the target instance ID after the target request is processed.

[0115] When the number of remaining requests exceeds the configured processing rate, it is determined that the current access volume of the target instance ID has reached the rate limit of the target instance ID.

[0116] In some alternative implementations, the speed limiting processing module 703 is further configured to:

[0117] If the target node with the target instance ID as the key does not exist in the shared memory area, then a new node is created in the shared memory area with the target instance ID as the key and the initial instance rate limiting information corresponding to the target instance ID as the value, and the rate limiting processing is applied to the target request.

[0118] In summary, this embodiment achieves direct rate limiting for Tengine instances by developing an instance specification rate limiting module. This allows rate limiting to be applied directly to user instances, rather than based on listeners. It monitors user instance information and, through configuration, distributes the instance ID and instance specification to the underlying data plane. Thus, when processing user requests, the corresponding rate limiting specification can be found based on the instance ID, and the request is rate-limited accordingly. This ensures that even if the number of listeners changes, the overall rate limiting of instances remains effectively controlled. Because this embodiment directly rates-limits user instances, rather than specific listeners, once the user confirms the instance specification, the rate limiting will remain unchanged regardless of any changes in the number of listeners. This makes rate limiting more accurate and efficient.

[0119] This invention also provides a computer device; please refer to [link / reference]. Figure 8 , Figure 8 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 8 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 8 Take a processor 10 as an example.

[0120] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0121] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0122] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0123] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0124] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0125] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0126] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0127] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and all such modifications and variations fall within the defined scope.

Claims

1. A method for instance rate limiting based on shared memory, characterized in that, The method is executed by the instance specification rate limiting module, which is set in the nginx system architecture. The nginx system architecture also has a shared memory area connected to the instance specification rate limiting module. The method includes: When a target request is received, the target listener accessed by the target request is identified by the destination IP address and destination port number of the target request. Obtain the target instance ID to which the target listener belongs from the configuration items of the instance specification rate limiting module; Based on the target instance ID, a target node with the target instance ID as the key is searched in the shared memory area, and the target request is rate-limited based on the instance rate-limiting information in the target node; the shared memory area is used to store each instance ID and the instance rate-limiting information corresponding to each instance ID.

2. The method according to claim 1, characterized in that, The configuration items of the instance specification rate limiting module include a first configuration item and a second configuration item. The first configuration item is used to indicate the instance ID to which each configured listener belongs, and the second configuration item is used to indicate the instance specification information corresponding to each instance ID.

3. The method according to claim 2, characterized in that, Obtaining the target instance ID to which the target listener belongs from the configuration items of the instance specification rate limiting module includes: Obtain the target instance ID to which the target listener belongs from the first configuration item of the instance specification rate limiting module; Obtain the target instance specification information corresponding to the target instance ID from the second configuration item of the instance specification rate limiting module.

4. The method according to any one of claims 1 to 3, characterized in that, The shared memory area organizes the instance IDs and the rate limit information of the user instance corresponding to each instance ID using a lookup tree data structure. Each node of the search tree corresponds one-to-one with each instance ID, and each node includes a key identifier and a value identifier. The key identifier is used to store the instance ID of the corresponding node, and the value identifier is used to store the instance rate limiting information corresponding to the instance ID. The instance rate limiting information includes historical access time, historical access volume, and instance specification information.

5. The method according to claim 4, characterized in that, The rate-limiting process for the target request based on the instance rate-limiting information in the target node includes: If a target node with the target instance id as the key exists in the shared memory area, then the target instance rate limiting information corresponding to the target instance id is obtained from the value bit identifier of the target node; Based on the target instance rate limiting information, determine whether the current access volume of the target instance ID has reached the rate limiting limit of the target instance ID; If the current access volume of the target instance ID reaches the rate limit of the target instance ID, the target request is rejected and an HTTP status code is returned; If the current access volume of the target instance ID has not reached the rate limit of the target instance ID, then update the target instance rate limit information in the value identifier of the target node, and perform rate limit processing on the target request.

6. The method according to claim 5, characterized in that, The step of determining whether the current access volume of the target instance ID has reached the rate limit limit of the target instance ID based on the target instance rate limit information includes: Obtain the remaining unprocessed request count, configured processing rate, and time interval for processing requests corresponding to the target instance ID, and calculate the remaining number of requests for the target instance ID after the target request is processed; When the number of remaining requests exceeds the configured processing rate, it is determined that the current access volume of the target instance ID has reached the rate limit of the target instance ID.

7. The method according to claim 4, characterized in that, The rate-limiting process for the target request based on the instance rate-limiting information in the target node includes: If there is no target node in the shared memory area with the target instance id as the key, then a new node is created in the shared memory area with the target instance id as the key and the initial instance rate limiting information corresponding to the target instance id as the value, and the target request is rate-limited.

8. A shared memory-based instance rate limiting device, characterized in that, The device is executed by an instance specification rate limiting module, which is set in the nginx system architecture. The nginx system architecture also includes a shared memory area connected to the instance specification rate limiting module. The device includes: The target listener confirmation module is used to confirm the target listener accessed by the target request by using the destination IP address and destination port number of the target request when a target request is received. The target instance ID acquisition module is used to obtain the target instance ID to which the target listener belongs from the configuration items of the instance specification rate limiting module; The rate limiting module is used to search for target nodes with the target instance ID as the key in the shared memory area based on the target instance ID, and to perform rate limiting processing on the target request based on the instance rate limiting information in the target node; the shared memory area is used to store each instance ID and the instance rate limiting information corresponding to each instance ID.

9. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform an instance rate limiting method based on shared memory as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to execute any one of claims 1 to 7 of an instance rate limiting method based on shared memory.

Images