Example speed limiting method and device based on shared memory

Abstract

The invention relates to the technical field of wireless communication, in particular to an instance speed limiting method and device based on a shared memory, the method is executed by an instance specification speed limiting module, and the method comprises the steps that when a target request is received, a target monitor accessed by the target request is confirmed through a target IP address and a target port number of the target request; obtaining a target instance id to which the target monitor belongs from a configuration item of the instance specification speed limiting module; based on the target instance id, searching a target node taking the target instance id as a key in a shared memory area, and performing speed limiting processing on the target request based on instance speed limiting information in the target node; the shared memory area is used for storing each instance id and instance speed limit information corresponding to each instance id. According to the method, the speed is limited directly according to the user instance without being influenced by the change of the number of the monitors, so that the speed limiting accuracy and efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology, and in particular to a method and device for instance speed limiting based on shared memory. Background Art

[0002] When users use the load balancing service provided by a cloud service provider, they usually purchase a load balancing instance of the corresponding specifications on the cloud service provider's console. After the purchase is completed, the front end will send the user's configuration information to the control plane, and the control plane will send this configuration information to the Agent. The Agent is an application used to send configurations. It can organize the received configuration information and write it to the configuration file nginx.conf of nginx (a high-performance HTTP and reverse proxy server). Subsequently, nginx will read this configuration file, obtain the user's configuration information, and provide services to the user according to this configuration information.

[0003] Therefore, under the current design architecture, nginx, as the main carrier of the network's seven-layer load balancing service, is not aware of user information and instance specifications. This means that when providing services, nginx cannot directly limit the speed based on the user's instance specifications. However, cloud service providers need to limit the speed from the instance dimension to more accurately control user services and billing. The existing solution for instance specification speed limit is generally to add up the sum of the speed limit specifications of all listeners under the current instance. That is, the speed limit specifications of all listeners under an instance are accumulated and used as the speed limit specification of the instance.

[0004] However, the speed limit of instance specifications should be directly applied to the instance itself, rather than by accumulating the speed limit specifications of the listeners. Secondly, when users add or reduce listeners but do not change the purchased instance specifications, the allocation of speed limit specifications becomes very difficult. Because at this time, if the flow of the instance is still limited according to the sum of the speed limit specifications of the listeners, the speed limit will be inaccurate or fail to meet the needs of users. Summary of the invention

[0005] In view of this, the present invention provides an instance speed limiting method and device based on shared memory to solve the problem that the existing solution cannot directly limit the speed according to the user's instance specifications and lacks accuracy.

[0006] In a first aspect, the present invention provides an instance speed limiting method based on shared memory, the method is executed by an instance specification speed limiting module, the instance specification speed limiting module is arranged in an nginx system architecture, and a shared memory area connected to the instance specification speed limiting module is also arranged in the nginx system architecture, the method comprising:

[0007] When receiving a target request, confirming the target listener accessed by the target request through the destination IP address and destination port number of the target request;

[0008] Obtaining the target instance ID to which the target listener belongs from the configuration item of the instance specification rate limit module;

[0009] Based on the target instance id, a target node with the target instance id as a key is searched in the shared memory area, and the target request is speed-limited based on the instance speed-limit information in the target node; the shared memory area is used to store each instance id and the instance speed-limit information corresponding to each instance id.

[0010] The above scheme of the present invention realizes direct speed limit on Tengine instance by developing instance specification speed limit module, allowing speed limit to be directly applied to user instance instead of based on listener, being able to pay attention to the information of user instance, and sending instance id and instance specification to the underlying data plane through configuration, so that when processing user request, the corresponding speed limit specification can be found according to instance id, and the request can be speed limited, ensuring that the total speed limit of the instance can be effectively controlled even when the number of listeners changes. Since the present invention directly applies speed limit to user instance instead of specific listener, when the user confirms the instance specification, the speed limit will not change no matter how the number of listeners is changed. This makes speed limit processing more accurate and efficient.

[0011] In an optional implementation, the configuration items of the instance specification rate limit module include a first configuration item and a second configuration item, wherein the first configuration item is used to indicate the instance ID to which each configured listener belongs, and the second configuration item is used to indicate the instance specification information corresponding to each instance ID.

[0012] In an optional implementation, obtaining the target instance ID to which the target listener belongs from the configuration item of the instance specification rate limit module includes:

[0013] Obtaining, from the first configuration item of the instance specification rate limit module, the target instance ID to which the target listener belongs;

[0014] From the second configuration item of the instance specification rate limit module, obtain the target instance specification information corresponding to the target instance ID.

[0015] In an optional implementation, the shared memory area organizes the instance IDs and the user instance speed limit information corresponding to each instance ID in a search tree data structure;

[0016] Each node of the search tree corresponds one-to-one to each instance ID, and each node includes a key identifier and a value identifier. The key identifier is used to store the instance ID of the corresponding node, and the value identifier is used to store the instance speed limit information corresponding to the instance ID; the instance speed limit information includes historical access time, historical access volume and instance specification information.

[0017] In an optional implementation, the performing rate limiting processing on the target request based on the instance rate limiting information in the target node includes:

[0018] If a target node with the target instance id as the key exists in the shared memory area, obtaining the target instance speed limit information corresponding to the target instance id from the value bit identifier of the target node;

[0019] According to the target instance speed limit information, determine whether the current access volume of the target instance ID reaches the speed limit upper limit of the target instance ID;

[0020] If the current number of visits to the target instance ID reaches the upper limit of the target instance ID, the target request is rejected and an HTTP status code is returned;

[0021] If the current access volume of the target instance ID does not reach the upper speed limit of the target instance ID, the target instance speed limit information in the value bit identifier of the target node is updated, and the target request is speed-limited.

[0022] In an optional implementation, judging whether the current access volume of the target instance ID reaches the upper limit of the speed limit of the target instance ID according to the speed limit information of the target instance includes:

[0023] Obtain the remaining unprocessed requests corresponding to the target instance ID, the configured processing rate, and the time interval for processing requests, and calculate the number of remaining requests for the target instance ID after the target request is processed;

[0024] When the remaining number of requests exceeds the configured processing rate, it is determined that the current access volume of the target instance ID reaches the upper limit of the speed limit of the target instance ID.

[0025] In an optional implementation, the performing rate limiting processing on the target request based on the instance rate limiting information in the target node includes:

[0026] If there is no target node with the target instance id as the key in the shared memory zone, then in the shared memory zone, a corresponding new node is established with the target instance id as the key and the initial instance speed limit information corresponding to the target instance id as the value, and the target request is speed limited.

[0027] In a second aspect, the present invention provides an instance speed limiting device based on shared memory, the device is executed by an instance specification speed limiting module, the instance specification speed limiting module is arranged in an nginx system architecture, and a shared memory area connected to the instance specification speed limiting module is also arranged in the nginx system architecture, the device includes:

[0028] A target listener confirmation module is used to confirm the target listener accessed by the target request through the destination IP address and destination port number of the target request when receiving the target request;

[0029] A target instance ID acquisition module, used to acquire the target instance ID to which the target listener belongs from the configuration item of the instance specification rate limit module;

[0030] A speed limit processing module is used to search for a target node with the target instance id as a key in the shared memory area based on the target instance id, and to perform speed limit processing on the target request based on the instance speed limit information in the target node; the shared memory area is used to store each instance id and the instance speed limit information corresponding to each instance id.

[0031] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to execute an instance speed limiting method based on shared memory according to the first aspect or any corresponding embodiment thereof.

[0032] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute an instance speed limiting method based on shared memory according to the first aspect or any corresponding embodiment thereof.

[0033] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions, wherein the computer instructions are used to enable a computer to execute an instance speed limiting method based on shared memory according to the first aspect or any corresponding embodiment thereof.

[0034] The technical solution provided by the present invention may include the following beneficial effects:

[0035] The present invention realizes direct speed limit on Tengine instance by developing instance specification speed limit module, allowing speed limit to be performed directly on user instance instead of based on listener, being able to pay attention to the information of user instance, and sending instance id and instance specification to the underlying data plane through configuration, so that when processing user request, the corresponding speed limit specification can be found according to instance id, and the request can be speed limited, ensuring that the total speed limit of the instance can be effectively controlled even when the number of listeners changes. Since the present invention directly limits the speed on user instance instead of specific listener, when the user confirms the instance specification, the speed limit will not change no matter how the number of listeners is changed. This makes the speed limit processing more accurate and efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 is a flow chart of an instance speed limiting method based on shared memory according to an embodiment of the present invention;

[0038] Figure 2 is a flowchart of another instance speed limiting method based on shared memory according to an embodiment of the present invention;

[0039] Figure 3 is a schematic diagram of organizing each node in a shared memory area using a search tree data structure according to an embodiment of the present invention;

[0040] Figure 4 is a flowchart of another instance speed limiting method based on shared memory according to an embodiment of the present invention;

[0041] Figure 5 is a calculation diagram of the optimized leaky bucket algorithm according to an embodiment of the present invention;

[0042] Figure 6 is a schematic diagram of a logic of judging whether a limit is exceeded using an optimized leaky bucket algorithm according to an embodiment of the present invention;

[0043] Figure 7 is a structural block diagram of an example speed limiting device based on shared memory according to an embodiment of the present invention;

[0044] Figure 8 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0046] First, some key terms in the present invention are explained:

[0047] Tengine: is a high-performance server that is mainly used to provide load balancing services in the field of network services. Its infrastructure is the same as the open source architecture nginx, but it is more functional and more extensible. When processing network requests, Tengine can better allocate resources, improve service efficiency and reliability, and adapt to more complex network scenarios and business needs. For example, in a high-concurrency network environment, Tengine can more flexibly respond to a large number of user requests and ensure the stable operation of the service.

[0048] Load balancing instance: is a specific embodiment of a running load balancing service. To use the load balancing service, you must first create a load balancing instance. It is the actual running unit that implements the load balancing function and is responsible for reasonably distributing network requests to multiple backend servers to avoid performance problems on a single server due to excessive load, ensuring the efficient operation of the entire system.

[0049] Listener: In the seven-layer load balancing, the listener is the key part of implementing the load balancing service and is the entrance of the network request. Its main implementation form is that the operating system listens to a specific ip:port combination. For example, when a user enters the URL in the browser to visit a website, the request first reaches the listener, and the listener decides to forward the request to the target backend server for processing according to the pre-set rules. A load balancing instance needs at least one listener, and multiple listeners can handle different types of requests according to different rules.

[0050] Instance specifications: For each load balancing instance, the instance specifications are used to measure the degree of user use of the load balancing service. In the network seven-layer load balancing service, the QPS (user request per second) value is usually used as the measurement standard. Cloud service providers provide a variety of different specifications of load balancing instances for users to choose from based on different business needs and user scales.

[0051] Leaky Bucket Algorithm: A commonly used algorithm for rate limiting. Its main purpose is to control the rate at which data is injected into the network. In network services, if there is a sudden large number of user requests (burst traffic), the leaky bucket algorithm can process these requests in sequence at the set rate to avoid server crashes due to instantaneous high loads, thereby ensuring service stability and normal use by other users.

[0052] Agent: It is an application program used to send configurations, which can be compared to the configuration transmitter of nginx. Its main task is to pass relevant configuration information to nginx. When users make configuration changes to the load balancing service in the cloud service provider console, these configuration information will eventually reach the agent through the front-end and control plane. The agent will organize these configurations and write them into the nginx configuration file nginx.conf, so that nginx can provide corresponding services according to the new configuration.

[0053] According to an embodiment of the present invention, this embodiment provides an nginx system architecture that has a newly added instance specification rate limit module. The nginx system architecture also has a shared memory area connected to the instance specification rate limit module (ie, the lb_inst_qps module).

[0054] Further, user A initiates a target request through a browser, the goal is to access a certain service. User A's request first reaches the load balancer. In this embodiment, there are two load balancer instances, marked as Server A and Server B respectively. The role of the load balancer is to distribute requests to the back-end Nginx server cluster according to preset rules (such as polling, least connection, etc.). The console is the interface for users to configure and manage load balancing services. Users configure information such as listeners and instance specifications in the console. The control plane of the console is a component responsible for processing and managing configuration information. It receives configuration information from the console and sends this information to the agent. The agent is a configuration sender responsible for synchronizing the configuration information of the control plane to the Nginx server. The Nginx server uses the nginx.conf file to store configuration information. The configuration file contains information such as the server's IP address, port number, and back-end server list. The Nginx server cluster is a server group that actually processes user requests. Each Nginx server runs the Nginx software, which processes requests according to the configuration in the nginx.conf file. The back-end server is the server that ultimately provides services. The Nginx server forwards requests to the back-end server according to the configuration.

[0055] Furthermore, this embodiment introduces an instance specification speed limit module (lb_inst_qps module), that is, using the flexible third-party extension module architecture provided by the nginx system architecture, the lb_inst_qps module is developed, so that nginx can directly perceive user instances and instance specifications. Through the instance specification speed limit module, the speed of each user instance can be limited, which makes up for the current lack of nginx in this function, and can be widely used in the load balancing services of cloud network vendors.

[0056] Due to the original design architecture of nginx, there is currently a lack of instance speed limit function, and the native architecture of nginx does not pay attention to the instance parameter. Therefore, this embodiment introduces an instance specification speed limit module (lb_inst_qps module). This module needs to add some configuration items, the most critical of which are the lb_inst_id configuration item and the lb_inst_qps_rate configuration item. The lb_inst_id configuration item is used to indicate the instance id to which each configured listener belongs, and the lb_inst_qps_rate configuration item is used to indicate the specification of the instance.

[0057] In the implementation scheme of the instance specification speed limit module (lb_inst_qps module), this embodiment first applies for a shared memory area. In the shared memory area, the instance specification information and the current access volume are stored in the form of key-value, where the key is the instance id of each request, and the value is a structure that stores information such as the current access volume, access time, and instance specifications. Whenever a request comes in, the listener accessed by the request can be confirmed by the request destination IP address and destination port number. At the same time, according to the configuration reading of the instance specification speed limit module (lb_inst_qps module), the instance id to which the listener belongs can be obtained through the listener, and then the specific specifications of the instance id can be queried, and then the optimized leaky bucket algorithm can be used to limit the request. Thereby achieving the goal of limiting the speed of user requests through instances.

[0058] In summary, this embodiment realizes direct speed limiting of Tengine instances by developing an instance specification speed limiting module, allowing speed limiting to be performed directly on user instances instead of based on listeners, being able to pay attention to the information of user instances, and sending the instance id and instance specifications to the underlying data plane through configuration. In this way, when processing user requests, the corresponding speed limit specifications can be found according to the instance id, and the request can be speed limited, ensuring that the total speed limit of the instance can be effectively controlled even when the number of listeners changes. Since this embodiment directly limits the speed of user instances instead of specific listeners, once the user confirms the instance specifications, the speed limit will not change no matter how the number of listeners is changed. This makes speed limit processing more accurate and efficient.

[0059] According to an embodiment of the present invention, an example speed limiting method embodiment based on shared memory is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0060] In this embodiment, a method for instance speed limiting based on shared memory is provided. The method is executed by an instance specification speed limiting module. The instance specification speed limiting module is arranged in an nginx system architecture. A shared memory area connected to the instance specification speed limiting module is also arranged in the nginx system architecture. Figure 1 is a flow chart of a method for limiting the speed of an instance based on shared memory according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0061] Step S101, when a target request is received, a target listener accessed by the target request is confirmed through the destination IP address and destination port number of the target request.

[0062] Furthermore, in the process of processing network requests, this embodiment must first determine the target listener corresponding to the target request. Because the listener is the first contact point after the network request enters the system, only by clarifying the target listener can the instance information related to it be further obtained, thereby providing a basis for subsequent speed limit processing. When the system receives the target request, it will extract the destination IP address and destination port number from the message of the target request. In the load-balanced nginx system architecture, each listener is associated with a specific IP address and port number. Through this mapping relationship, the target listener accessed by the target request can be confirmed.

[0063] Step S102: Obtain the target instance ID to which the target listener belongs from the configuration item of the instance specification rate limit module.

[0064] Furthermore, the purpose of obtaining the target instance ID to which the target listener belongs is a key step in associating the listener with a specific instance. Only by knowing the target instance ID can we further obtain information such as the speed limit rules of the instance in order to accurately process the request. This embodiment searches from the configuration items of the instance specification speed limit module (i.e., the lb_inst_qps module). These configuration items have been loaded during the previous initialization process, which contain the correspondence between the listener and the instance ID. By searching in these configuration items through the relevant identification or configuration information of the target listener, the target instance ID to which it belongs can be obtained.

[0065] Step S103, based on the target instance id, search the shared memory area for a target node with the target instance id as the key, and perform speed limit processing on the target request based on the instance speed limit information in the target node; the shared memory area is used to store each instance id and the instance speed limit information corresponding to each instance id.

[0066] Furthermore, this embodiment finds the corresponding target node in the shared memory area according to the acquired target instance ID, and then uses the instance speed limit information in the target node to speed limit the target request. The shared memory area stores each instance ID and related instance speed limit information. By quickly finding the target node, the speed limit information can be efficiently obtained and processed to achieve accurate speed limit for the request, ensuring that the entire system runs stably within the instance specification range.

[0067] This embodiment first searches in a shared memory area, which may store information in the form of a red-black tree plus a hash table, wherein the nodes of the red-black tree use the instance id as the key, and the value of the node is a structure containing instance speed limit information (such as access volume, access time, instance specifications, etc.). After finding the target node with the target instance id as the key, this embodiment determines whether the target request exceeds the speed limit based on the instance speed limit information in the node (combined with the leaky bucket algorithm, etc.). If the request volume does not exceed the speed limit, the request is allowed to continue processing, and the access volume and other information in the node are updated (such as adding 1 to the access volume, updating the access time, etc.); if the speed limit is exceeded, the request is rejected and the corresponding HTTP status code is returned.

[0068] In summary, this embodiment realizes direct speed limiting of Tengine instances by developing an instance specification speed limiting module, allowing speed limiting to be performed directly on user instances instead of based on listeners, being able to pay attention to the information of user instances, and sending the instance id and instance specifications to the underlying data plane through configuration. In this way, when processing user requests, the corresponding speed limit specifications can be found according to the instance id, and the request can be speed limited, ensuring that the total speed limit of the instance can be effectively controlled even when the number of listeners changes. Since this embodiment directly limits the speed of user instances instead of specific listeners, once the user confirms the instance specifications, the speed limit will not change no matter how the number of listeners is changed. This makes speed limit processing more accurate and efficient.

[0069] In this embodiment, another instance speed limiting method based on shared memory is provided. The method is executed by an instance specification speed limiting module. The instance specification speed limiting module is set in the nginx system architecture. The nginx system architecture is also provided with a shared memory area connected to the instance specification speed limiting module. Figure 2 is a flow chart of another instance speed limiting method based on shared memory according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0070] Step S201: When a target request is received, the target listener accessed by the target request is confirmed through the target IP address and the target port number of the target request. Figure 2 Step S201 of the illustrated embodiment will not be described in detail here.

[0071] Step S202: Obtain the target instance ID to which the target listener belongs from the configuration item of the instance specification rate limit module.

[0072] In an optional implementation, the configuration items of the instance specification rate limit module include a first configuration item and a second configuration item, the first configuration item is used to indicate the instance ID to which each configured listener belongs, and the second configuration item is used to indicate the instance specification information corresponding to each instance ID.

[0073] In an optional implementation, step S202 includes:

[0074] Obtaining the target instance ID to which the target listener belongs from the first configuration item of the instance specification rate limit module;

[0075] From the second configuration item of the instance specification rate limit module, obtain the target instance specification information corresponding to the target instance ID.

[0076] Furthermore, the configuration items of the instance specification rate limit module contain key information. The first configuration item (lb_inst_id configuration item) establishes an association between the listener and the instance id to which it belongs, and the second configuration item (lb_inst_qps_rate configuration item) clarifies the instance specification information corresponding to each instance id. These two configuration items work together to enable the system to accurately obtain the detailed specifications of the instance to which the target listener belongs, providing a basis for subsequent rate limit operations. In this embodiment, first, from the first configuration item, according to the identifier or related configuration content of the target listener, find the target instance id to which it belongs. Then, from the second configuration item, the corresponding target instance specification information is obtained through the obtained target instance id. For example, if the first configuration item records that listener A belongs to instance 1, then it is determined that the target instance id is 1, and then the instance specification information corresponding to instance 1 is searched in the second configuration item, such as the maximum number of requests allowed per second (QPS value).

[0077] Furthermore, this embodiment combines the third-party module extension interface provided by nginx to develop an instance specification speed limit module (i.e., lb_inst_qps module). This module is developed based on the nginx third-party extension structure, adds configuration items and processing modules, and intervenes in the access stage of nginx request processing. From the implementation point of view, the instance specification speed limit module (i.e., lb_inst_qps module) first adds structures such as ngx_http_lb_inst_qps_node_t and ngx_http_lb_inst_qps_limit_t, which respectively represent the red-black tree node information used to store speed limit information and the shared memory block storing speed limit information and the specific speed limit value, and then adds some lb_inst_qps_zone (initialize speed limit shared memory block), lb_inst_qps_limit (specify specific speed limit value), lb_inst_qps_status (users can customize the return code of nginx when reaching the speed limit value to distinguish it from other abnormal situations), and some subsequent functions such as searching for user instance id and judging whether the user has reached the speed limit value.

[0078] That is to say, as a commonly used server software, the Nginx system architecture provides a third-party module expansion interface. This embodiment uses this feature to develop an instance specification speed limit module (i.e., lb_inst_qps module), which enables specific functions to be added to the nginx core architecture without changing it, and enhances nginx's ability in instance speed limit to meet the demand for accurate speed limit of load balancing instances in cloud services. The instance specification speed limit module (i.e., lb_inst_qps module) intervenes in the access phase of nginx processing requests. At this stage, the instance specification speed limit module (i.e., lb_inst_qps module) can obtain relevant information of the request, such as the destination IP address and port number, thereby providing an opportunity for instance-based speed limit operations. The gx_http_lb_inst_qps_node_t structure is used to represent the red-black tree node information for subsequent storage of speed limit information. The red-black tree is a self-balancing binary search tree that can provide efficient search, insertion, and deletion operations when processing large amounts of data. Through this structure, the module can effectively organize and manage instance speed limit-related data in the red-black tree, making it easy to quickly locate and update the speed limit information of a specific instance. The ngx_http_lb_inst_qps_limit_t structure is used to represent the shared memory block that stores speed limit information and the specific speed limit value. The use of shared memory can improve data access speed, reduce data replication overhead, and enable multiple processes or threads to quickly access and modify speed limit information. This structure clarifies the format and related information of speed limit values ​​stored in shared memory to ensure accurate storage and reading of speed limit values. The instance specification speed limit module (i.e., lb_inst_qps module) also adds a series of functions, such as the function of finding the user instance id, which is used to determine the instance to which the request belongs when processing a request; the function of determining whether the user has reached the speed limit value, which determines whether to allow the request to continue processing by calculating and comparing the request amount with the speed limit value. These functions work together with the newly added structures and instructions to implement the complete process from obtaining request information, finding the instance id, determining the speed limit situation to finally processing the request (accepting or rejecting), ensuring the accurate implementation of the instance-based speed limit function.

[0079] Step S203: Based on the target instance id, search the shared memory area for a target node with the target instance id as a key. The shared memory area is used to store each instance id and instance speed limit information corresponding to each instance id.

[0080] In an optional implementation, the shared memory area organizes the instance IDs and the user instance speed limit information corresponding to each instance ID in a search tree data structure;

[0081] Each node of the search tree corresponds one-to-one to each instance ID, and each node includes a key identifier and a value identifier. The key identifier is used to store the instance ID of the corresponding node, and the value identifier is used to store the instance speed limit information corresponding to the instance ID; the instance speed limit information includes historical access time, historical access volume and instance specification information.

[0082] Furthermore, the shared memory area of ​​the present embodiment adopts a data structure of a search tree (such as a red-black tree) to organize each instance ID and its corresponding instance speed limit information. This structure helps to quickly search and manage data and improve system performance. Each node of the search tree corresponds to an instance ID, the key identifier stores the instance ID, and the value identifier stores the instance speed limit information including historical access time, historical access volume, and instance specification information. Based on the target instance ID, the present embodiment searches in the search tree of the shared memory area, starting from the root node, and traverses the tree nodes step by step according to the comparison between the key identifier and the target instance ID until the target node whose key identifier matches the target instance ID is found. For example, if the target instance ID is 5, starting from the root node of the search tree, the size relationship between the node's key identifier and 5 is compared, and the search continues to the left or right subtree until the target node with a key of 5 is found.

[0083] Furthermore, the instance specification rate limit module (i.e., lb_inst_qps module) needs to first load the lb_inst_id configuration item and lb_inst_qps_rate configuration item in the configuration file during initialization, and then apply for a shared memory block for subsequent storage of instance id information and the corresponding access volume for each id as well as instance specification information. In the shared memory, user instance information is actually stored in the form of a red-black tree plus a hash table. For details, see Figure 3 The diagram shows that the shared memory area organizes each node in a search tree data structure. The lb_inst_qps module maintains a red-black tree, in which each node of the red-black tree stores information in the form of key-value, where the key is the user instance id (such as Figure 3 lb_id1 to lb_id5), value (such as Figure 3 The qps_node in the __name__ is a complex structure that contains the time when the user instance was last accessed, the instance specifications, and other information. Initially, the red-black tree is empty. Using a red-black tree plus a hash table to store data can significantly improve query efficiency. Even if the number of instances is large enough, the search efficiency will not be significantly reduced. After completing the above operations, the instance specification rate limit module (i.e., the lb_inst_qps module) will wait for the request to arrive.

[0084] That is to say, when the module is initialized, loading the lb_inst_id configuration item and lb_inst_qps_rate configuration item in the configuration file is the key first step. The lb_inst_id configuration item is used to clarify the instance id to which each configured listener belongs, and is the basis for subsequently associating the listener with the instance. For example, in a complex network architecture, there may be multiple listeners serving different instances, and this configuration item can accurately know the instance corresponding to each listener. The lb_inst_qps_rate configuration item indicates the specification of the instance, which is usually measured by the QPS (user request per second) value, and determines the upper limit of the ability of the instance to process requests. Applying for a shared memory block is to efficiently store instance-related information. Shared memory allows multiple processes or modules to quickly access the same memory area, avoiding frequent copying of data in different memory spaces, thereby improving data access speed. In this embodiment, it is necessary to store instance id information, the number of visits corresponding to each id, and instance specification information, etc. in the future. This information needs to be frequently read and updated when processing requests. Using shared memory can significantly improve system performance. In shared memory, user instance information is stored in the form of a red-black tree plus a hash table. Each node of the red-black tree is stored in the form of key-value, where the key is the user instance id, which allows the corresponding node to be quickly located by the instance id. The value is a complex structure that contains important information such as the last access time of the user instance and the instance specifications. For example, when a request arrives, the system can quickly find the corresponding node in the red-black tree by the instance id, and then obtain information such as the last access time and instance specifications in the node structure to determine whether to limit the speed and how to limit the speed.

[0085] Step S204, if there is a target node with the target instance id as the key in the shared memory area, obtain the target instance speed limit information corresponding to the target instance id from the value bit identifier of the target node; based on the target instance speed limit information, determine whether the current access volume of the target instance id reaches the speed limit upper limit of the target instance id.

[0086] In an optional implementation, the step S304 includes:

[0087] Obtain the remaining unprocessed requests corresponding to the target instance ID, the configured processing rate, and the time interval for processing requests, and calculate the number of remaining requests for the target instance ID after the target request is processed;

[0088] When the remaining number of requests exceeds the configured processing rate, it is determined that the current access volume of the target instance ID reaches the upper limit of the speed limit of the target instance ID.

[0089] Furthermore, if the target node is found in the shared memory area, this embodiment obtains the target instance speed limit information from its value bit identifier, including historical access time, historical access volume and instance specification information (such as configured processing rate, etc.). Then the number of remaining requests for the target instance id after the target request is processed is calculated. The calculation formula is based on the optimized leaky bucket algorithm idea to determine whether the speed limit is reached: the calculated number of remaining requests is compared with the configured processing rate. When the number of remaining requests exceeds the configured processing rate, it is determined that the current access volume of the target instance id has reached the speed limit. This means that if the request continues to be processed, it may exceed the processing capacity of the instance and affect system stability.

[0090] Further, when a user request comes, please refer to Figure 4 The flowchart of another instance rate limiting method based on shared memory is shown. In this embodiment, the listener that the request wants to access can be confirmed according to the destination IP address and destination port number of each request message. Then the instance specification rate limiting module (i.e., lb_inst_qps module) will search the instance id and instance specification to which the listener belongs in the configuration. After obtaining the above information, the instance specification rate limiting module (i.e., lb_inst_qps module) will search the shared memory for the instance id and find out whether there is a node with the instance id as the key in the current red-black tree. If so, it means that the instance has been visited. Then, the value of the node is obtained. According to the information in the value, combined with the leaky bucket algorithm, it is calculated whether the access volume of the instance id has reached the upper limit. If the upper limit is reached, the request is rejected and the HTTP status code is returned; if the upper limit is not reached, the request is accepted, and the access time in the red-black tree node is updated, and the access volume is increased by 1. Then, the control is returned to nginx, and nginx continues to perform subsequent processing on the request. If no node with the instance id as the key is found in the red-black tree, it means that the instance is being accessed for the first time. In this case, a new red-black tree node is added with the instance id as the key, and the current access time is recorded. The initial access volume is 0, and the access volume is now increased by 1. The node is inserted into the red-black tree, and then the control is returned to nginx, which then processes the request.

[0091] Step S205: If the current number of visits to the target instance ID reaches the upper speed limit of the target instance ID, the target request is rejected and an HTTP status code is returned; if the current number of visits to the target instance ID does not reach the upper speed limit of the target instance ID, the target instance speed limit information in the value bit identifier of the target node is updated, and the target request is speed limited.

[0092] Furthermore, if it is determined that the current access volume of the target instance ID reaches the upper speed limit, the system rejects the target request and returns an HTTP status code to inform the client that the request is restricted. This is to protect system resources and prevent overload. If the upper speed limit is not reached, the target instance speed limit information in the value bit identifier of the target node is updated, such as updating the historical access time to the current time and adding 1 to the historical access volume. The target request is then processed at a speed limit, that is, the request is forwarded to the backend server for processing according to the normal process, but it will be limited by the instance specifications to ensure that the overall system operates within the specified load range.

[0093] Step S206, if there is no target node with the target instance id as the key in the shared memory zone, then in the shared memory zone, a corresponding new node is established with the target instance id as the key and the initial instance speed limit information corresponding to the target instance id as the value, and the target request is speed limited.

[0094] Furthermore, if the target node with the target instance id as the key is not found in the shared memory area, it means that the instance is accessed for the first time. At this time, this embodiment uses the target instance id as the key in the shared memory area and the initial instance speed limit information corresponding to the target instance id (such as the initial access volume is 0, the initial access time is the current time, and the instance specification information is obtained from the configuration item) as the value to establish a new node. This prepares for the subsequent request processing and speed limit statistics of the instance, and then the target request is speed-limited, the request is forwarded to the backend server, and the information such as the access volume of the instance is counted and managed.

[0095] Furthermore, this embodiment is implemented based on the idea of ​​leaky bucket algorithm, but compared with the traditional leaky bucket algorithm, this embodiment has made some designs different from the traditional leaky bucket algorithm in combination with current business needs. First of all, the whole is implemented based on the leaky bucket algorithm, so the main calculation formula is based on the basic principle of the leaky bucket algorithm. The following is the main judgment basis, please refer to Figure 5 The calculation diagram of the optimized leaky bucket algorithm is shown in the figure, where lr represents the red-black tree node currently hit by lb_id, lr->excess represents the total number of unprocessed requests remaining in the red-black tree node, rate represents the request processing rate configured by the customer in the configuration file, ms represents the time interval from the last processing to this processing, in ms, and +1000 represents the request currently being processed (the value is initially 1, but for the convenience of calculation, the formula has been multiplied by 1000 as a whole, so here it is 1000 instead of 1). The above formula can be used to calculate the number of requests remaining after the request is processed. When this number exceeds the remaining processing capacity, the bucket overflows, and the requests received will be rejected because the upper limit of the speed limit is reached. Please refer to Figure 6The schematic diagram of the logic of judging whether the limit is exceeded after the optimized leaky bucket algorithm is shown. The judgment condition is that if the number of remaining requests is greater than the rate, the excess requests will be rejected. The traditional leaky bucket algorithm allows for sudden increase in traffic. The judgment logic is whether the remaining request amount is greater than the allowed burst amount, that is, the limit->burst parameter. In the leaky bucket algorithm optimized in this embodiment, when the user request amount reaches the threshold, access will be directly denied. In theory, if burst is not allowed, limt->burst can be set to 0. However, after deduction and verification, this threshold setting cannot be 0, but must be rate. That is, when the limit value is high, greater than 1000qps (that is, more than one request must be processed in 1ms) and the requests are concentrated in a certain time window, if the threshold is set to 0, the difference in processing time between two requests is less than 1ms. From the formula, the stock excess value cannot be reduced, so this embodiment needs to set the excess value to rate, which is equivalent to artificially amortizing the requests that may be concentrated to each time scale of the unit time window. If the threshold is 0, it is equivalent to only calculating the request value in the current surge time window, and the calculated speed limit value is low. Therefore, the threshold is set to rate, which is actually rate*1000ms / 1000, that is, the total number of requests that can be processed within 1s (1000ms) (that is, the qps unit calculation time is 1s). When the number of remaining requests to be processed is greater than this value, it means that the bucket is overflowing, and new requests will be rejected because they have reached the upper limit of the leaky bucket algorithm.

[0096] In summary, this embodiment realizes direct speed limiting of Tengine instances by developing an instance specification speed limiting module, allowing speed limiting to be performed directly on user instances instead of based on listeners, being able to pay attention to the information of user instances, and sending the instance id and instance specifications to the underlying data plane through configuration. In this way, when processing user requests, the corresponding speed limit specifications can be found according to the instance id, and the request can be speed limited, ensuring that the total speed limit of the instance can be effectively controlled even when the number of listeners changes. Since this embodiment directly limits the speed of user instances instead of specific listeners, once the user confirms the instance specifications, the speed limit will not change no matter how the number of listeners is changed. This makes speed limit processing more accurate and efficient.

[0097] In the present embodiment, a shared memory-based example speed limiting device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0098] This embodiment provides an instance speed limiting device based on shared memory, such as Figure 7 As shown, including:

[0099] The target listener confirmation module 701 is used to confirm the target listener accessed by the target request through the destination IP address and destination port number of the target request when receiving the target request;

[0100] The target instance ID acquisition module 702 is used to acquire the target instance ID to which the target listener belongs from the configuration item of the instance specification rate limit module;

[0101] The speed limit processing module 703 is used to search for a target node with the target instance id as the key in the shared memory area based on the target instance id, and perform speed limit processing on the target request based on the instance speed limit information in the target node; the shared memory area is used to store each instance id and the instance speed limit information corresponding to each instance id.

[0102] In some optional implementations, the configuration items of the instance specification rate limit module include a first configuration item and a second configuration item, the first configuration item being used to indicate the instance ID to which each configured listener belongs, and the second configuration item being used to indicate the instance specification information corresponding to each instance ID.

[0103] In some optional implementations, the target instance ID acquisition module 702 is further used to:

[0104] Obtaining the target instance ID to which the target listener belongs from the first configuration item of the instance specification rate limit module;

[0105] From the second configuration item of the instance specification rate limit module, obtain the target instance specification information corresponding to the target instance ID.

[0106] In some optional implementations, the shared memory area organizes the instance IDs and the user instance speed limit information corresponding to each instance ID in a search tree data structure;

[0107] Each node of the search tree corresponds one-to-one to each instance ID, and each node includes a key identifier and a value identifier. The key identifier is used to store the instance ID of the corresponding node, and the value identifier is used to store the instance speed limit information corresponding to the instance ID; the instance speed limit information includes historical access time, historical access volume and instance specification information.

[0108] In some optional implementations, the speed limit processing module 703 is further used to:

[0109] If there is a target node with the target instance id as the key in the shared memory zone, the target instance speed limit information corresponding to the target instance id is obtained from the value bit identifier of the target node;

[0110] According to the speed limit information of the target instance, determine whether the current access volume of the target instance ID reaches the speed limit upper limit of the target instance ID;

[0111] If the current number of visits to the target instance ID reaches the upper limit of the target instance ID, the target request is rejected and an HTTP status code is returned;

[0112] If the current access volume of the target instance ID does not reach the upper speed limit of the target instance ID, the target instance speed limit information in the value bit identifier of the target node is updated, and the target request is speed-limited.

[0113] In some optional implementations, the speed limit processing module 703 is further used to:

[0114] Obtain the remaining unprocessed requests corresponding to the target instance ID, the configured processing rate, and the time interval for processing requests, and calculate the number of remaining requests for the target instance ID after the target request is processed;

[0115] When the remaining number of requests exceeds the configured processing rate, it is determined that the current access volume of the target instance ID reaches the upper limit of the speed limit of the target instance ID.

[0116] In some optional implementations, the speed limit processing module 703 is further used to:

[0117] If there is no target node with the target instance id as the key in the shared memory zone, then in the shared memory zone, a corresponding new node is established with the target instance id as the key and the initial instance speed limit information corresponding to the target instance id as the value, and the target request is speed limited.

[0118] In summary, this embodiment realizes direct speed limiting of Tengine instances by developing an instance specification speed limiting module, allowing speed limiting to be performed directly on user instances instead of based on listeners, being able to pay attention to the information of user instances, and sending the instance id and instance specifications to the underlying data plane through configuration. In this way, when processing user requests, the corresponding speed limit specifications can be found according to the instance id, and the request can be speed limited, ensuring that the total speed limit of the instance can be effectively controlled even when the number of listeners changes. Since this embodiment directly limits the speed of user instances instead of specific listeners, once the user confirms the instance specifications, the speed limit will not change no matter how the number of listeners is changed. This makes speed limit processing more accurate and efficient.

[0119] The embodiment of the present invention also provides a computer device, see Figure 8 , Figure 8 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 8 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 8 A processor 10 is taken as an example.

[0120] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0121] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0122] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0123] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0124] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0125] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0126] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0127] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the defined scope.

Claims

1. A method for limiting the speed of an instance based on shared memory, characterized in that: The method is performed by an instance specification rate limiting module, the instance specification rate limiting module is arranged in an nginx system architecture, and a shared memory area connected to the instance specification rate limiting module is also arranged in the nginx system architecture, and the method includes: When receiving a target request, confirming the target listener accessed by the target request through the destination IP address and destination port number of the target request; Obtaining the target instance ID to which the target listener belongs from the configuration item of the instance specification rate limit module; Based on the target instance id, a target node with the target instance id as a key is searched in the shared memory area, and the target request is speed-limited based on the instance speed-limit information in the target node; the shared memory area is used to store each instance id and the instance speed-limit information corresponding to each instance id.

2. The method according to claim 1, characterized in that The configuration items of the instance specification rate limit module include a first configuration item and a second configuration item, the first configuration item is used to indicate the instance id to which each configured listener belongs, and the second configuration item is used to indicate the instance specification information corresponding to each instance id.

3. The method according to claim 2, characterized in that The step of obtaining the target instance ID to which the target listener belongs from the configuration item of the instance specification rate limit module includes: Obtaining, from the first configuration item of the instance specification rate limit module, the target instance ID to which the target listener belongs; From the second configuration item of the instance specification rate limit module, obtain the target instance specification information corresponding to the target instance ID.

4. The method according to any one of claims 1 to 3, characterized in that: The shared memory area organizes the instance IDs and the user instance speed limit information corresponding to each instance ID in a search tree data structure; Each node of the search tree corresponds one-to-one to each instance ID, and each node includes a key identifier and a value identifier. The key identifier is used to store the instance ID of the corresponding node, and the value identifier is used to store the instance speed limit information corresponding to the instance ID; the instance speed limit information includes historical access time, historical access volume and instance specification information.

5. The method according to claim 4, characterized in that The performing rate limiting processing on the target request based on the instance rate limiting information in the target node includes: If a target node with the target instance id as the key exists in the shared memory area, obtaining the target instance speed limit information corresponding to the target instance id from the value bit identifier of the target node; According to the target instance speed limit information, determine whether the current access volume of the target instance ID reaches the speed limit upper limit of the target instance ID; If the current number of visits to the target instance ID reaches the upper limit of the target instance ID, the target request is rejected and an HTTP status code is returned; If the current access volume of the target instance ID does not reach the upper speed limit of the target instance ID, the target instance speed limit information in the value bit identifier of the target node is updated, and the target request is speed-limited.

6. The method according to claim 5, characterized in that The determining, according to the target instance rate limit information, whether the current access volume of the target instance ID reaches the rate limit upper limit of the target instance ID includes: Obtain the remaining unprocessed requests corresponding to the target instance ID, the configured processing rate, and the time interval for processing requests, and calculate the number of remaining requests for the target instance ID after the target request is processed; When the remaining number of requests exceeds the configured processing rate, it is determined that the current access volume of the target instance ID reaches the upper limit of the speed limit of the target instance ID.

7. The method according to claim 4, characterized in that The performing rate limiting processing on the target request based on the instance rate limiting information in the target node includes: If there is no target node with the target instance id as the key in the shared memory zone, then in the shared memory zone, a corresponding new node is established with the target instance id as the key and the initial instance speed limit information corresponding to the target instance id as the value, and the target request is speed limited.

8. An instance speed limiting device based on shared memory, characterized in that: The device is executed by an instance specification speed limit module, the instance specification speed limit module is set in the nginx system architecture, and the nginx system architecture is also provided with a shared memory area connected to the instance specification speed limit module, and the device includes: A target listener confirmation module is used to confirm the target listener accessed by the target request through the destination IP address and destination port number of the target request when receiving the target request; A target instance ID acquisition module, used to acquire the target instance ID to which the target listener belongs from the configuration item of the instance specification rate limit module; A speed limit processing module is used to search for a target node with the target instance id as a key in the shared memory area based on the target instance id, and to perform speed limit processing on the target request based on the instance speed limit information in the target node; the shared memory area is used to store each instance id and the instance speed limit information corresponding to each instance id.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the instance speed limiting method based on shared memory as described in any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute an instance speed limiting method based on shared memory according to any one of claims 1 to 7.

Images