Interface request processing method and device, server and server cluster
By dynamically adjusting the concurrency threshold of interface requests and utilizing Redis caching technology, the performance degradation caused by a fixed concurrency level in the server cluster was resolved, thereby improving the server's processing capacity and resource utilization efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING WODONG TIANJUN INFORMATION TECH CO LTD
- Filing Date
- 2021-04-29
- Publication Date
- 2026-05-15
AI Technical Summary
Existing interface rate limiting technology, which sets a fixed number of concurrent connections in a server cluster, leads to a decrease in server performance, especially when third-party services are unavailable or time out, resulting in severe resource consumption.
By acquiring real-time interface performance and dynamically adjusting the concurrency threshold, and utilizing Redis caching technology, we can avoid relying on third-party maximum concurrency limits and achieve rate limiting.
It improves server performance, avoids resource consumption caused by sudden increases in concurrency or capacity expansion, and reduces reliance on third-party services.
Smart Images
Figure CN115269643B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of server technology, and in particular to a method, apparatus, server, and server cluster for processing interface requests. Background Technology
[0002] Interface rate limiting refers to protecting the server system by limiting the rate of concurrent access requests or the number of access requests within a time window. When the concurrent access requests reach the limit rate or the limit number, the access request is rejected or queued.
[0003] In existing technologies, the main method of interface rate limiting is to set a fixed limit on the number of concurrent requests. If the number of concurrent access requests exceeds the limit, the interface rate limiting will be implemented.
[0004] However, existing interface rate limiting technologies are mainly applied to single servers, with each server having a fixed limit on the number of concurrent connections. As the number of servers increases, the fixed limit on the number of concurrent connections will exceed the maximum number of concurrent connections provided by the third party. This makes the server system mainly dependent on the maximum number of concurrent connections provided by the third party. Once the third-party service becomes unavailable or the call times out, it will cause a large number of access requests to occupy the server system's resources, resulting in a decrease in server performance. Summary of the Invention
[0005] This application provides a method, apparatus, server, and server cluster for processing interface requests, which addresses the problem that the fixed concurrency limit of existing server interfaces easily leads to a decline in server performance.
[0006] In a first aspect, embodiments of this application provide a method for processing interface requests, applied to a server, comprising:
[0007] Get the current concurrency count of the target interface and the interface performance in the previous time period, where the current concurrency count represents the number of interface requests in a concurrent state during the current time period;
[0008] Based on the preset configuration data and the interface performance, the current concurrency threshold cached in the preset database is updated to obtain the updated current concurrency threshold.
[0009] If the current concurrency is greater than or equal to the updated current concurrency threshold, then rate limiting is applied to the interface requests of the target interface.
[0010] In one possible design of the first aspect, the preset configuration data includes a first preset threshold and a second preset threshold. The step of updating the current concurrency threshold cached in the preset database based on the preset configuration data and the interface performance to obtain the updated current concurrency threshold includes:
[0011] If the interface performance is less than or equal to the first preset threshold, the current concurrency threshold cached in the preset database is increased to obtain the updated current concurrency threshold.
[0012] If the interface performance is greater than the first preset threshold and also greater than the second preset threshold, then the current concurrency threshold cached in the preset database is lowered to obtain the updated current concurrency threshold.
[0013] In another possible design of the first aspect, the preset configuration data further includes a preset concurrency upper limit and a preset concurrency increase step. If the interface performance is less than or equal to a first preset threshold, the current concurrency threshold cached in the preset database is increased to obtain an updated current concurrency threshold, including:
[0014] If the interface performance is less than or equal to the first preset threshold, then the current minute concurrency threshold is calculated based on the preset concurrency upper limit, the preset concurrency increase step, and the current concurrency threshold cached in the preset database. The current minute concurrency threshold is used to represent the maximum number of interface requests in a concurrent state within the current time period.
[0015] Based on the current concurrency threshold per minute, the current concurrency threshold cached in the preset database is increased to obtain the updated current concurrency threshold.
[0016] In another possible design of the first aspect, the step of increasing the current concurrency threshold cached in the preset database based on the current minute concurrency threshold to obtain the updated current concurrency threshold includes:
[0017] The current concurrency threshold per minute is assigned to the current concurrency threshold cached in the preset database to obtain the adjusted current concurrency threshold.
[0018] Compare the adjusted current concurrency threshold with the preset concurrency limit;
[0019] If the adjusted current concurrency threshold is greater than the preset concurrency limit, then the preset concurrency limit is assigned to the adjusted current concurrency threshold to obtain the updated current concurrency threshold.
[0020] If the adjusted current concurrency threshold is less than or equal to the preset concurrency limit, then the adjusted current concurrency threshold is used as the updated current concurrency threshold.
[0021] In another possible design of the first aspect, the preset configuration data further includes a preset interface circuit breaker concurrency threshold. After lowering the current concurrency threshold cached in the preset database to obtain the updated current concurrency threshold, the method further includes:
[0022] The updated current concurrency threshold is compared with the preset interface circuit breaker concurrency threshold;
[0023] If the updated current concurrency threshold is less than the interface circuit breaker concurrency threshold, then the target interface will be circuit breaker-broken.
[0024] In another possible design of the first aspect, before lowering the current concurrency threshold of the target interface to obtain the updated current concurrency threshold, the method further includes:
[0025] If the interface performance is greater than the first preset threshold and less than the second preset threshold, then the lifetime of the current minute concurrency threshold cached in the preset database is updated.
[0026] In another possible design of the first aspect, before obtaining the current concurrency of the target interface and the interface performance in the previous time period, it further includes:
[0027] Read the preset database;
[0028] If the preset database caches a circuit breaker flag, then the target interface will be circuit breaker triggered.
[0029] In another possible design of the first aspect, the preset configuration data further includes a preset initial concurrency count. Before updating the current concurrency threshold cached in the preset database based on the preset configuration data and the interface performance to obtain the updated current concurrency threshold, the following steps are also included:
[0030] The preset initial concurrency number is assigned to the current concurrency threshold cached in the preset database.
[0031] In another possible design of the first aspect, before obtaining the current concurrency of the target interface and the interface performance in the previous time period, it further includes:
[0032] The target interface is obtained by intercepting the server's interface using aspect-oriented programming.
[0033] In another possible design of the first aspect, obtaining the interface performance of the target interface in the previous time period includes:
[0034] Get the total time taken for the target interface to complete the interface request and the total number of times the target interface completed the interface request in the previous time period;
[0035] Based on the total time consumed and the total number of times, obtain the interface performance of the target interface in the previous time period.
[0036] In another possible design of the first aspect, the preset database is a remote dictionary service.
[0037] Secondly, embodiments of this application provide an interface request processing apparatus, comprising:
[0038] The acquisition module is used to acquire the current concurrency count of the target interface and the interface performance in the previous time period. The current concurrency count represents the number of interface requests in a concurrent state during the current time period.
[0039] The update module is used to update the current concurrency threshold cached in the preset database according to the preset configuration data and the interface performance, so as to obtain the updated current concurrency threshold.
[0040] The processing module is used to perform rate limiting processing on the interface requests of the target interface if the current concurrency is greater than or equal to the updated current concurrency threshold.
[0041] Thirdly, embodiments of this application provide a server, including: a memory and at least one processor;
[0042] The memory stores computer-executed instructions;
[0043] The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the method as described above.
[0044] Fourthly, embodiments of this application provide a server cluster, including at least one server as described above.
[0045] Fifthly, embodiments of this application provide a readable storage medium having a computer program stored thereon, the readable storage medium storing computer instructions which, when executed by a processor, are used to implement the method as described above.
[0046] Sixthly, embodiments of this application provide a computer program product, including a computer program / instructions that, when executed by a processor, implement the method described above.
[0047] The interface request processing method, apparatus, server, and server cluster provided in this application embodiment obtain the performance of the server interface in the previous time period in real time and update and adjust the concurrency threshold for the current time period. This makes the server no longer dependent on the maximum concurrency provided by a third party. When the concurrency of interface requests reaches the updated and adjusted concurrency threshold, rate limiting can be performed to avoid excessive access requests from consuming system resources and improve the service performance of the server. Attached Figure Description
[0048] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application;
[0049] Figure 1 A schematic diagram illustrating a scenario for the interface request processing method provided in this application embodiment;
[0050] Figure 2 A flowchart illustrating an embodiment of the interface request processing method provided in this application.
[0051] Figure 3 A flowchart illustrating a second embodiment of the interface request processing method provided in this application.
[0052] Figure 4 A flowchart illustrating Embodiment 3 of the interface request processing method provided in this application;
[0053] Figure 5 A flowchart illustrating Embodiment 4 of the interface request processing method provided in this application;
[0054] Figure 6 A flowchart illustrating Embodiment 5 of the interface request processing method provided in this application;
[0055] Figure 7 An interface diagram of an embodiment of an interface request processing apparatus provided in this application;
[0056] Figure 8 A schematic diagram of the structure of the interface request processing device according to Embodiment 2 of this application;
[0057] Figure 9 A schematic diagram of the structure of the interface request processing device according to Embodiment 3 provided in this application;
[0058] Figure 10 This is a schematic diagram of the server structure provided in an embodiment of this application.
[0059] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation
[0060] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0061] First, let me explain the terms used in this application:
[0062] Redis:
[0063] Redis (Remote Dictionary Service) is an open-source, C-language-written, network-enabled, in-memory or persistent log-structured key-value database.
[0064] Traffic limiting:
[0065] In this embodiment, rate limiting refers to limiting the access requests, denying service, blocking, and queuing them when the concurrent access requests to the interface reach a threshold.
[0066] Figure 1 This is a schematic diagram illustrating a scenario for the interface request processing method provided in an embodiment of this application. For example... Figure 1 As shown, this embodiment includes a server cluster 10 and a terminal device 11 that communicates with the server cluster 10 via a network. The server cluster 10 consists of one or more servers. The terminal device 11 initiates an access request to the server via the network to obtain the interface data of the target interface. The total number of access requests initiated by the terminal device 11 to the server cluster 10 within a time period (e.g., one minute) can be recorded as the current concurrency.
[0067] In practical applications, terminal devices 11 include mobile phones, computers, tablets, etc. When a terminal device 11 initiates an access request is uncontrollable, which causes the current concurrency to fluctuate in different time periods. For example, when a product is launched online for the first time, a large number of order requests will be generated, or when a game is released, a large number of login requests will be generated. In order to ensure the service performance and avoid these requests from consuming a lot of system resources, the server can limit the rate of these requests.
[0068] Currently, the main types of rate limiting technologies include the following:
[0069] 1. Concurrency control, implemented through semaphore mechanisms such as Semaphore in Java, controls the number of concurrent requests to the server by pre-configuring the maximum concurrency. However, concurrency control is primarily applied to single servers. As the cluster expands, the rate-limited concurrency increases linearly, which can easily lead to the interface's concurrency limit exceeding the maximum concurrency provided by a third party. Here, "third party" can refer to an external interface that provides services to the application.
[0070] 2. Leaky Bucket Algorithm: Given a number of accessible requests per unit time, the leaky bucket algorithm limits the number of concurrent requests. While it can limit the maximum average transmission rate, its configuration is fixed and it is not suitable for sudden increases in the number of concurrent requests.
[0071] 3. Token Bucket Algorithm: The system sends tokens to the token bucket at a constant rate. Token release stops when the bucket is full. Requests can only be executed if a token is acquired. The token bucket algorithm can support validation for sudden increases in concurrent requests, but it only limits the rate of access for a single service. As the cluster expands, the interface's concurrency limit may exceed the maximum concurrency provided by a third party.
[0072] In summary, existing rate limiting technologies mainly rely on third-party services. If the third-party service is unavailable or other reasons cause the call to time out, the request will consume a large amount of system resources and affect the performance of other services; dynamic adjustment of rate limiting thresholds is not supported.
[0073] To address the aforementioned issues, this application provides a method, apparatus, server, and server cluster for processing interface requests. By determining the interface performance and dynamically adjusting the current concurrency threshold based on that performance, rate limiting can be implemented to prevent excessive interface calls from consuming server resources. Furthermore, by caching the current concurrency threshold in a database using Redis caching technology, all servers in the system can directly retrieve the adjusted threshold from the database, eliminating reliance on third-party maximum concurrency limits. This avoids the problem of third-party service unavailability leading to excessive system resource consumption and improves service performance.
[0074] Figure 2 This is a flowchart illustrating an embodiment of the interface request processing method provided in this application. This method can be applied to a server, such as... Figure 2 As shown, the method for processing this interface request may include the following steps:
[0075] S201. Obtain the current concurrency of the target interface and the interface performance in the previous time period.
[0076] The current concurrency count represents the number of interface requests that are in a concurrent state within the current time period. For example, the time period can be one minute.
[0077] In this embodiment, interface performance refers to the average time spent accessing the target interface or the average time spent calling the target interface. For example, the target interface refers to a software interface, which may be an interface with a large number of access requests.
[0078] For example, interface performance can be calculated based on the total number of calls and total time taken by the target interface within a time period. The current concurrency level, the total number of calls and total time taken by the target interface within a time period can be stored in corresponding cache variables in a preset database. Specifically, cache variables T1, T2, and N1 can be defined in the preset database, where T1, T2, and N1 are all string structures. T1 represents the total time taken by the target interface within a time period, with the key being the target interface's identifier and timestamp. T1 is valid in the preset database for two time periods: the previous time period and the current time period. T2 represents the total number of calls by the target interface within a time period, with the key being the target interface's identifier and timestamp. T2 is valid in the preset database for two time periods. N1 represents the current concurrency level, with the key being the target interface's identifier. N1 is valid indefinitely in the preset database.
[0079] S202. Based on the preset configuration data and interface performance, update the current concurrency threshold cached in the preset database to obtain the updated current concurrency threshold.
[0080] Specifically, a cache variable N2 is set in the preset database. N2 is a string structure and represents the current concurrency threshold. The key value of N2 is the identifier of the target interface. Its validity period in the preset database lasts for one time period. That is, when N2 is updated in the current time period, N2 will continue to be updated in the next time period to achieve dynamic updates.
[0081] In this embodiment, the preset configuration data can be pre-configured, specifically including the upper limit of concurrency C1, the initial concurrency C2, the interface performance threshold P1 for increasing concurrency, the number of steps for increasing concurrency S, the interface performance threshold P2 for decreasing concurrency, the interface circuit breaker concurrency threshold D, and the circuit breaker duration T. These configured data are constants, and their values will not change.
[0082] Among them, the upper limit of concurrency C1, the initial concurrency C2, the interface performance threshold P1 for increasing concurrency, the number of steps for increasing concurrency S, and the interface performance threshold P2 for decreasing concurrency are used as initial data. Based on the size of the interface performance, different calculation formulas are used to substitute these initial data into the corresponding calculation formulas to obtain the calculation results to update the current concurrency threshold.
[0083] The concurrency threshold D and circuit breaker duration T can be used as the criteria for determining the target interface to be circuit-broken. When the target interface is circuit-broken, access and requests to that target interface will no longer be permitted.
[0084] For example, when the service is first started, the N2 value cached in the database is set to C2. Subsequently, the value of N2 will be dynamically updated according to the preset configuration data and interface performance.
[0085] S203. If the current concurrency is greater than or equal to the updated current concurrency threshold, then rate limiting is applied to the interface requests of the target interface.
[0086] Specifically, rate limiting includes measures such as limiting the rate of access requests, denying service, blocking, and queuing.
[0087] In this embodiment, when the service starts, the target interface will continuously receive interface requests. When the concurrency of interface requests is low, the interface business logic will be executed without rate limiting the interface requests. Taking one minute as an example, at the beginning of one minute, the interface performance of the target interface in the previous minute is obtained first, and the current concurrency threshold is dynamically adjusted according to the interface performance. If the number of interface requests in a concurrent state received by the target interface in the current minute (i.e., the current concurrency) is greater than or equal to the dynamically adjusted current concurrency threshold, rate limiting can also be applied to these interface requests.
[0088] This application embodiment calculates the interface performance of the target interface and dynamically adjusts the current concurrency threshold based on the interface performance. This allows each server in the server system to directly retrieve the cached current concurrency threshold from the preset database and perform rate limiting based on the concurrency threshold. This enables the server to adapt to sudden increases in concurrency, improves server performance, and ensures that the server system is not affected by server expansion. It also avoids the maximum concurrency of interface calls exceeding the maximum concurrency provided by third-party interfaces due to expansion, thus reducing dependence on third-party services.
[0089] Based on the above embodiments, in some embodiments, if the preset configuration data includes a first preset threshold and a second preset threshold, then step S202 can be implemented through the following steps:
[0090] If the interface performance is less than or equal to the first preset threshold, the current concurrency threshold cached in the preset database will be increased to obtain the updated current concurrency threshold.
[0091] If the interface performance is greater than the first preset threshold and also greater than the second preset threshold, then the current concurrency threshold cached in the preset database will be lowered to obtain the updated current concurrency threshold.
[0092] Specifically, the first preset threshold can be the interface performance threshold P1 for increasing concurrency, and the second preset threshold can be the interface performance threshold P2 for decreasing concurrency. The interface performance can be calculated based on T1 and T2.
[0093] For example, the interface performance can be T1 / T2. When (T1 / T2) <= P1, the value of the current concurrency threshold N2 is increased. When (T1 / T2) > P1 and is greater than P2, the value of the current concurrency threshold N2 is decreased.
[0094] This application embodiment determines the adjustment direction of the current concurrency threshold by comparing the interface performance with the first preset threshold and the second preset threshold, and dynamically adjusts the current concurrency threshold so that the target interface can cope with sudden increases in concurrency, avoid excessive number of interface calls in sudden situations, and avoid consuming server resources. At the same time, when the interface performance allows, the current concurrency threshold can be increased to improve the service performance of the server.
[0095] Based on the above embodiments, in some embodiments, if the preset configuration data includes the above-mentioned preset concurrency limit C1 and preset concurrency increase step S, then the above step "if the interface performance is less than or equal to the first preset threshold, then the current concurrency threshold cached in the preset database is increased to obtain the updated current concurrency threshold" can be implemented through the following steps:
[0096] If the interface performance is less than or equal to the first preset threshold, the current minute concurrency threshold is calculated based on the preset concurrency limit, the preset concurrency increase step, and the current concurrency threshold cached in the preset database.
[0097] Based on the current concurrency threshold per minute, the current concurrency threshold cached in the preset database is increased to obtain the updated current concurrency threshold.
[0098] The current minute concurrency threshold is used to represent the maximum number of interface requests that are in a concurrent state within the current time period.
[0099] Specifically, in addition to caching variables such as T1, T2, N1, and N2, the preset database can also cache the current minute concurrency threshold N3. N3 can be a string structure, with the key value being the identifier of the target interface and a timestamp, used to represent the current minute concurrency threshold. The validity period of N3 in the preset database is one time period. If the value of N3 becomes invalid in the next time period, it will be invalid.
[0100] For example, the formula for calculating the current minute concurrency threshold N3 can be: N3 = N2 + C1 / S.
[0101] The current concurrency threshold N2 can be increased to: N2 = N3.
[0102] In this embodiment, the values of the current concurrency threshold N2 and the current minute concurrency threshold N3 cannot exceed the preset concurrency limit C1. Optionally, before adjusting the current concurrency threshold N2, it can be determined whether the current concurrency threshold N2 is less than the preset concurrency limit C1. If it is less than the preset concurrency limit C1, the current concurrency threshold N2 can be adjusted upwards. If it is equal to the preset concurrency limit C1, the current concurrency threshold N2 will not be adjusted upwards.
[0103] This application embodiment assigns a current minute concurrency threshold value and caches it in a preset database. Within the current time period, the current concurrency threshold can be dynamically adjusted based on the current minute concurrency threshold. The current minute concurrency threshold remains valid within the current time period and becomes invalid in the next time period, requiring recalculation. This allows the current concurrency threshold to be dynamically adjusted within different time periods, thereby effectively responding to various emergencies that occur at different times and improving service performance.
[0104] Based on the above embodiments, in some embodiments, the step of "adjusting the current concurrency threshold cached in the preset database upwards according to the current minute concurrency threshold to obtain the updated current concurrency threshold" can be specifically implemented through the following steps:
[0105] The current concurrency threshold per minute is assigned to the current concurrency threshold cached in the preset database to obtain the adjusted current concurrency threshold.
[0106] Compare the adjusted current concurrency threshold with the preset concurrency limit;
[0107] If the current concurrency threshold after the increase is greater than the preset concurrency limit, then the preset concurrency limit is assigned to the current concurrency threshold after the increase, and the updated current concurrency threshold is obtained.
[0108] If the adjusted current concurrency threshold is less than or equal to the preset concurrency limit, then the adjusted current concurrency threshold will be used as the updated current concurrency threshold.
[0109] Specifically, before assigning the current concurrent number N3 to the current concurrent number N2, the current concurrent number threshold N2 cached in the preset database can be set to the initial concurrent number C2. After N3 is assigned to the current concurrent number threshold N2 cached in the preset database, the adjusted current concurrent number threshold N2 = N3 = N2 + C1 / S is obtained.
[0110] This application embodiment obtains an increased current concurrency threshold by assigning the current minute concurrency threshold to the current concurrency threshold, and then limits the increased current concurrency threshold to avoid the increased current concurrency threshold exceeding the preset concurrency limit, which would cause the interface request concurrency to be too high, resulting in a large amount of service resources being occupied and affecting server performance.
[0111] Based on the above embodiments, Figure 3 The flowchart of Embodiment 2 of the interface request processing method provided in this application is shown below. Figure 3 As shown, the method for processing this interface request may also include the following steps:
[0112] S301. Obtain the current concurrency of the target interface and the interface performance in the previous time period.
[0113] S302. If the interface performance is greater than the first preset threshold and also greater than the second preset threshold, then the current concurrency threshold cached in the preset database is lowered to obtain the updated current concurrency threshold.
[0114] S303. Compare the updated current concurrency threshold with the preset interface circuit breaker concurrency threshold.
[0115] S304. If the updated current concurrency threshold is less than the interface circuit breaker concurrency threshold, then the target interface will be circuit breaker-broken.
[0116] For an explanation of step S301, please refer to step S201 above. Here, only steps S302, S303 and S304 will be explained.
[0117] Specifically, the preset configuration data includes the aforementioned preset interface circuit breaker concurrency threshold D. For example, the preset configuration data also includes the aforementioned circuit breaker duration T. When the updated current concurrency threshold N2 is less than the interface circuit breaker concurrency threshold D, the target interface will be circuit-broken for duration T. When the target interface is circuit-broken, an error message will be returned directly, and interface requests / accesses will no longer be accepted.
[0118] In this embodiment, the current concurrency threshold N2 cached in the preset database can be taken as the starting concurrency C2. When the interface performance is greater than the first preset threshold P1 and greater than the second preset threshold P2, N2 / 2 can be used as the updated current concurrency threshold N2.
[0119] Optionally, before comparing the updated current concurrency threshold N2 with the interface circuit breaker concurrency threshold, the updated current concurrency threshold N2 can be assigned to the current minute concurrency threshold N3 cached in the preset database.
[0120] This application embodiment compares the updated current concurrency threshold with the interface circuit breaker concurrency threshold. When the updated current concurrency threshold is too small, it indicates that the current interface performance of the target interface is poor and there may be a timeout. By circuit breaking the target interface, it is possible to prevent the interface from becoming unavailable due to timeout and thus occupying service resources.
[0121] Based on the above embodiments, Figure 4 The flowchart of Embodiment 3 of the interface request processing method provided in this application is shown below. Figure 4 As shown, the method for processing this interface request may also include the following steps:
[0122] S401. Obtain the current concurrency of the target interface and the interface performance in the previous time period.
[0123] S402. If the interface performance is greater than the first preset threshold and less than or equal to the second preset threshold, then update the lifetime of the current minute concurrency threshold cached in the preset database.
[0124] The explanation of step S401 can be found in step S201 above; only step S402 will be explained here. Specifically, the lifetime, i.e., the validity period of the current minute concurrency threshold N3 in the preset database, indicates that if the interface performance is greater than the first preset threshold and less than or equal to the second preset threshold, it means that the interface performance of the target interface in the previous time period is within the normal range, i.e., there is no need to adjust the current concurrency threshold upwards or downwards. By updating the lifetime of the current minute concurrency threshold N3, the server can determine that no rate limiting is needed based on the current minute concurrency threshold N3.
[0125] In this embodiment of the application, the interface performance is compared with a first preset threshold and a second preset threshold. When the interface performance is greater than the first preset threshold and less than or equal to the second preset threshold, it means that the current concurrency threshold does not need to be adjusted temporarily. At the same time, the current minute concurrency threshold is updated so that the server can effectively determine whether to perform rate limiting on the target interface.
[0126] Based on the above embodiments, Figure 5 The flowchart of Embodiment 4 of the interface request processing method provided in this application is shown below. Figure 5 As shown, the method for processing this interface request may also include the following steps:
[0127] S501. Read the preset database. If the preset database has a circuit breaker flag cached, then circuit breaker the target interface.
[0128] S502. If the circuit breaker flag is not cached in the preset database, obtain the current concurrency of the target interface and the interface performance in the previous time period.
[0129] S503. Based on the preset configuration data and interface performance, update the current concurrency threshold cached in the preset database to obtain the updated current concurrency threshold.
[0130] S504. If the current concurrency is greater than or equal to the updated current concurrency threshold, then rate limiting is applied to the interface requests of the target interface.
[0131] Specifically, the cached variables in the preset database include the circuit breaker flag DF, as well as the total time T1 for the target interface to complete calls within a time period, the total number of calls T2 for the target interface within a time period, the current concurrency N1, the current concurrency threshold N2, and the current minute concurrency threshold N3. The circuit breaker flag DF can be a string structure, with its key value being the target interface identifier. The circuit breaker flag DF represents degradation, meaning the target interface is circuit-broken, causing it to directly return an error message and block the interface request. The validity period of the circuit breaker flag DF in the preset database is the aforementioned circuit breaker duration T.
[0132] This application embodiment caches circuit breaker identifiers in a preset database. When the server determines that a circuit breaker identifier exists in the preset database, it can directly trip the target interface corresponding to the circuit breaker identifier, thereby avoiding the occupation of service resources due to service timeout exceptions of the target interface and reducing the impact on other service interfaces.
[0133] Based on the above embodiments, in some embodiments, before step S201, the following steps are further included:
[0134] Aspect-oriented programming is used to intercept the server's interface and obtain the target interface.
[0135] In this embodiment, Aspect-Oriented Programming (AOP) is a technique that achieves unified maintenance of program functions through pre-compilation and runtime dynamic proxies. AOP can isolate different parts of business logic, thereby reducing the coupling between different parts of business logic and improving the reusability of the program.
[0136] This application embodiment utilizes AOP technology to enable performance monitoring of specific interfaces of a server. When the interface performance decreases or increases, the server can dynamically adjust the current concurrency threshold of that interface to improve the processing efficiency of interface requests.
[0137] Based on the above embodiments, in some embodiments, the step S201 of "obtaining the interface performance of the target interface in the previous time period" can be specifically implemented through the following steps:
[0138] Get the total time taken to complete the target interface request and the total number of times the target interface completed the interface request in the previous time period;
[0139] Based on the total time and total number of times, obtain the interface performance of the target interface in the previous time period.
[0140] In this embodiment, the total time T1 for the target interface to complete the call in the previous time period and the total number of calls T2 for the target interface to complete the call in the previous time period can be determined. The interface performance of the target interface in the previous time period can be calculated by dividing T1 by T2.
[0141] This application embodiment determines the total time T1 for the target interface to complete the call in the previous time period and the total number of calls T2 for the target interface to complete the call in the previous time period. This allows for the rapid acquisition of the interface performance of each server interface and facilitates the dynamic adjustment of the current concurrency threshold for each interface.
[0142] Optionally, in some embodiments, the aforementioned preset database can be a remote dictionary service.
[0143] Optional, Figure 6 The flowchart of Embodiment 5 of the interface request processing method provided in this application is shown below. Figure 6 As shown, it includes steps S600 to S628.
[0144] In this embodiment, the preset configuration data includes the upper limit of concurrency C1, the initial concurrency C2, the interface performance threshold P1 for increasing concurrency, the number of steps for increasing concurrency S, the interface performance threshold P2 for decreasing concurrency, the interface circuit breaker concurrency threshold D, and the circuit breaker duration T. The preset database is set with cached variables, specifically including the total time T1 for the target interface to complete the call within a time period, the total number of times the target interface to complete the call within a time period T2, the current concurrency N1, the current concurrency threshold N2, the current minute concurrency threshold N3, and the circuit breaker flag DF.
[0145] Among them, step S600, whether the circuit breaker indicator DF exists.
[0146] Specifically, when the circuit breaker flag DF is assigned a value in the preset database, it means that the circuit breaker flag DF exists; when the circuit breaker flag DF is not assigned a value, it means that the circuit breaker flag DF does not exist.
[0147] Does step S601, N3 exist?
[0148] Specifically, the cached variable N3 in the preset database summary represents the current minute concurrency threshold. If the current minute concurrency threshold N3 is assigned a value, it means that N3 exists; if N3 is not assigned a value, it means that N3 does not exist.
[0149] In this embodiment, since the validity period of the current minute concurrency threshold N3 is one time period, taking a time period of one minute as an example, the current minute concurrency threshold N3 does not exist during the first request of each minute, and remains in existence from the second request of each minute onwards.
[0150] Step S602: Cache N1 and execute incr+1.
[0151] Specifically, by determining the current concurrency N1 of the target interface, N1 is cached in a preset database, and incr+1 indicates that N1 = N1+1.
[0152] S603. Obtain N2. Specifically, extract N2 from the preset database, which can be Redis.
[0153] S604. Are N2 empty? Specifically, if N2 is not assigned a value in the preset database, it means that N2 is empty; if it is assigned a value, then N2 is determined to be not empty, and the assigned value is extracted.
[0154] S605, N2 = C2, specifically, assign the value of C2 to N2. S606, Determine if N1 > N2.
[0155] S607. Cache N1 and execute incr-1. Specifically, cache the value of N1 at this time in a preset database, and incr-1 means making N1 = N1 - 1.
[0156] S608. Interface request rate limiting. Specifically, if N1 > N2, the server performs rate limiting on this interface.
[0157] S609. Execute the interface business logic. Specifically, when N1 <= N2, the target interface returns corresponding interface data according to the interface request to execute the interface business logic.
[0158] S610. Obtain the execution time T3. Specifically, the time spent by the interface when executing the interface business logic is the execution time T3.
[0159] S611. T1 = T1 + T3. Specifically, after the interface executes a business logic once, T1 increases by T3 on the basis of its original value.
[0160] S612. T2 = T2 + 1. Specifically, every time the interface processes an interface request, T2 increases by 1 on the basis of its original value. S613. Cache N1 and execute incr-1.
[0161] S614. Determine whether T2 and N2 exist. Specifically, at the first request after the service runs, the current concurrency threshold N2 does not exist, and the starting concurrency C2 is assigned to the current concurrency threshold N2, and N2 exists in subsequent requests.
[0162] S615. P1 = T1 / T2. S616. Determine whether PT <= P1. S617. N3 = N2 + C1 / S. S618. Determine whether N2 < C1. S619. N2 = N3. Specifically, assign N3 to N2. S620. Determine whether N2 > C1. S621. N2 = C1. Specifically, assign C1 to N2. S622. Determine whether PT > P2. S623. Update the expiration time of N3. Specifically, N3 is cached in a preset database, and the expiration time refers to the time N3 survives in the preset database. S624. N2 = N2 / 2. Specifically, reduce the value of N2 by half. S625. N3 = N2. Specifically, assign N2 to N3. S626. Determine whether N2 < D. S627. Assign DF, delete N2, T1, and T2. S628. Interface fusing. Specifically, fuse this target interface.
[0163] In summary, this application embodiment can dynamically adjust the current concurrency threshold of an interface based on its performance. When the number of concurrent requests to an interface is too high, rate limiting can be implemented to improve the interface's flexibility. Conversely, when the interface performance is too low, the interface can be directly circuit-broken, effectively preventing the interface performance from impacting the server and other interfaces. Furthermore, the dynamically adjusted current concurrency threshold is cached in Redis, and its corresponding key-value pair can identify the target interface. Each server in the server system can directly retrieve the current concurrency threshold of a specific interface from Redis and use this threshold for rate limiting. This prevents the current concurrency threshold from exceeding the maximum concurrency threshold provided by the third-party interface when the server system is scaled up, reducing reliance on third-party services and avoiding the problem of interface requests consuming a large amount of system resources and affecting service performance due to the unavailability or timeout of third-party services.
[0164] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.
[0165] Figure 7 This is an interface diagram of an embodiment of the interface request processing device provided in this application. The processing device can be integrated into the server, or it can work independently of the server and in cooperation with the server to implement the technical solution of this application.
[0166] For example, such as Figure 7 As shown, the interface request processing device 70 includes an acquisition module 71, an update module 72, and a processing module 73. The acquisition module 71 acquires the current concurrency count of the target interface and its performance in the previous time period. The update module 72 updates the current concurrency threshold cached in a preset database based on preset configuration data and interface performance, obtaining the updated current concurrency threshold. The processing module 73 performs rate limiting on the interface request of the target interface if the current concurrency count is greater than or equal to the updated current concurrency threshold.
[0167] The current concurrency count represents the number of API requests that are in a concurrent state within the current time period.
[0168] Optionally, in some embodiments, if the preset configuration data includes a first preset threshold and a second preset threshold, then the above-mentioned update module 72 can be specifically used for:
[0169] If the interface performance is less than or equal to the first preset threshold, the current concurrency threshold cached in the preset database will be increased to obtain the updated current concurrency threshold.
[0170] If the interface performance is greater than the first preset threshold and also greater than the second preset threshold, then the current concurrency threshold cached in the preset database will be lowered to obtain the updated current concurrency threshold.
[0171] Optionally, in some embodiments, if the preset configuration data also includes a preset concurrency limit and a preset concurrency increase step, then the above-mentioned update module 72 can be specifically used for:
[0172] If the interface performance is less than or equal to the first preset threshold, the current minute concurrency threshold is calculated based on the preset concurrency limit, the preset concurrency increase step, and the current concurrency threshold cached in the preset database. The current minute concurrency threshold is used to represent the maximum number of interface requests in a concurrent state within the current time period.
[0173] Based on the current concurrency threshold per minute, the current concurrency threshold cached in the preset database is increased to obtain the updated current concurrency threshold.
[0174] In some embodiments, the above-mentioned update module 72 can be specifically used for:
[0175] The current concurrency threshold per minute is assigned to the current concurrency threshold cached in the preset database to obtain the adjusted current concurrency threshold.
[0176] Compare the adjusted current concurrency threshold with the preset concurrency limit;
[0177] If the current concurrency threshold after the increase is greater than the preset concurrency limit, then the preset concurrency limit is assigned to the current concurrency threshold after the increase, and the updated current concurrency threshold is obtained.
[0178] If the adjusted current concurrency threshold is less than or equal to the preset concurrency limit, then the adjusted current concurrency threshold will be used as the updated current concurrency threshold.
[0179] Figure 8 This is a schematic diagram of a second embodiment of the interface request processing device provided in this application. The preset configuration data also includes a preset interface circuit breaker concurrency threshold, such as... Figure 8 As shown, the processing device 80 for the interface request includes an acquisition module 81, an update module 82, a processing module 83, and a circuit breaker module 84. The explanations of the acquisition module 81, update module 82, and processing module 83 can be found in the above embodiments, and will not be repeated here.
[0180] The circuit breaker module 84 is specifically used to compare the updated current concurrency threshold with the preset interface circuit breaker concurrency threshold; if the updated current concurrency threshold is less than the interface circuit breaker concurrency threshold, then the target interface is circuit breaker-broken.
[0181] Figure 9 A schematic diagram of the structure of the interface request processing device according to Embodiment 3 provided in this application is shown below. Figure 9 As shown, the processing device 90 for the interface request may include an acquisition module 91, an update module 92, a processing module 93, a circuit breaker module 94, and a caching module 95. The explanations of the acquisition module 91, update module 92, processing module 93, and circuit breaker module 94 can be found in the above embodiments. The caching module 95 is used to update the lifetime of the current minute concurrency threshold cached in the preset database if the interface performance is greater than a first preset threshold and less than or equal to a second preset threshold.
[0182] In some embodiments, the processing device for the above-mentioned interface request may further include an identification module for reading a preset database; if a circuit breaker identifier is cached in the preset database, the target interface is circuit breaker-broken.
[0183] In some embodiments, if the preset configuration data further includes a preset initial concurrency number, the processing device for the above-mentioned interface request may further include an assignment module for assigning the preset initial concurrency number to the current concurrency threshold cached in the preset database.
[0184] In some embodiments, the processing apparatus for the aforementioned interface request may further include an interception module for intercepting the server's interface using aspect-oriented programming to obtain the target interface.
[0185] Optionally, in some embodiments, the above-described acquisition module may be specifically used for:
[0186] Get the total time taken to complete the target interface request and the total number of times the target interface completed the interface request in the previous time period;
[0187] Based on the total time and total number of times, obtain the interface performance of the target interface in the previous time period.
[0188] Optionally, the aforementioned preset database can be a remote field service.
[0189] The apparatus provided in this application embodiment can be used to perform... Figures 2 to 6 The methods in the illustrated embodiments are similar in principle and technical effect, and will not be described again here.
[0190] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).
[0191] Optional, Figure 10 This is a schematic diagram of the server structure provided in the embodiments of this application, such as... Figure 10 As shown, the server includes a memory 101 and at least one processor 102. The memory 101 stores computer-executable instructions; the at least one processor 102 executes the computer-executable instructions stored in the memory 101, causing the at least one processor 102 to perform the method described above.
[0192] Optionally, embodiments of this application also provide a server cluster, including at least one of the servers described above.
[0193] Optionally, embodiments of this application also provide a readable storage medium storing a computer program thereon, wherein the readable storage medium stores computer instructions, which, when executed by a processor, are used to implement the method as described above.
[0194] Optionally, embodiments of this application also provide a computer program product, including a computer program / instructions that, when executed by a processor, implement the above-described method.
[0195] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates an "or" relationship between the preceding and following related objects; in formulas, the character " / " indicates a "division" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.
[0196] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application. In the embodiments of this application, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0197] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.
Claims
1. A method for processing interface requests, characterized in that, Applied to servers, including: Get the current concurrency of the target interface and the interface performance in the previous time period. The current concurrency represents the number of interface requests in the current time period that are in a concurrent state. The interface performance in the previous time period is determined based on the total time taken for the target interface to complete the interface requests and the total number of times the target interface completed the interface requests in the previous time period. Based on the preset configuration data and the interface performance, the current concurrency threshold cached in the preset database is updated to obtain the updated current concurrency threshold. If the current concurrency is greater than or equal to the updated current concurrency threshold, then rate limiting is applied to the interface requests of the target interface. The preset configuration data includes a first preset threshold and a second preset threshold. The first preset threshold is an interface performance threshold for increasing concurrency, and the second preset threshold is an interface performance threshold for decreasing concurrency. The step of updating the current concurrency threshold cached in the preset database based on the preset configuration data and the interface performance to obtain the updated current concurrency threshold includes: If the interface performance is less than or equal to the first preset threshold, the current concurrency threshold cached in the preset database is increased to obtain the updated current concurrency threshold. If the interface performance is greater than the first preset threshold and also greater than the second preset threshold, then the current concurrency threshold cached in the preset database is lowered to obtain the updated current concurrency threshold.
2. The method according to claim 1, characterized in that, The preset configuration data also includes a preset concurrency limit and a preset concurrency increase step. If the interface performance is less than or equal to a first preset threshold, the current concurrency threshold cached in the preset database is increased to obtain an updated current concurrency threshold, including: If the interface performance is less than or equal to the first preset threshold, then the current minute concurrency threshold is calculated based on the preset concurrency upper limit, the preset concurrency increase step, and the current concurrency threshold cached in the preset database. The current minute concurrency threshold is used to represent the maximum number of interface requests in a concurrent state within the current time period. Based on the current concurrency threshold per minute, the current concurrency threshold cached in the preset database is increased to obtain the updated current concurrency threshold.
3. The method according to claim 2, characterized in that, The step of adjusting the current concurrency threshold cached in the preset database upwards based on the current minute concurrency threshold to obtain the updated current concurrency threshold includes: The current concurrency threshold per minute is assigned to the current concurrency threshold cached in the preset database to obtain the adjusted current concurrency threshold. Compare the adjusted current concurrency threshold with the preset concurrency limit; If the adjusted current concurrency threshold is greater than the preset concurrency limit, then the preset concurrency limit is assigned to the adjusted current concurrency threshold to obtain the updated current concurrency threshold. If the adjusted current concurrency threshold is less than or equal to the preset concurrency limit, then the adjusted current concurrency threshold is used as the updated current concurrency threshold.
4. The method according to claim 1, characterized in that, The preset configuration data also includes a preset interface circuit breaker concurrency threshold. After lowering the current concurrency threshold cached in the preset database to obtain the updated current concurrency threshold, the process further includes: The updated current concurrency threshold is compared with the preset interface circuit breaker concurrency threshold; If the updated current concurrency threshold is less than the interface circuit breaker concurrency threshold, then the target interface will be circuit breaker-broken.
5. The method according to claim 1, characterized in that, Before lowering the current concurrency threshold of the target interface to obtain the updated current concurrency threshold, the method further includes: If the interface performance is greater than the first preset threshold and less than or equal to the second preset threshold, then the lifetime of the current minute concurrency threshold cached in the preset database is updated.
6. The method according to any one of claims 1-5, characterized in that, Before obtaining the current concurrency count and interface performance of the target interface in the previous time period, the method further includes: Read the preset database; If the preset database caches a circuit breaker flag, then the target interface will be circuit breaker triggered.
7. The method according to any one of claims 1-5, characterized in that, The preset configuration data also includes a preset initial concurrency count. Before updating the current concurrency threshold cached in the preset database based on the preset configuration data and the interface performance to obtain the updated current concurrency threshold, the process further includes: The preset initial concurrency number is assigned to the current concurrency threshold cached in the preset database.
8. The method according to any one of claims 1-5, characterized in that, Before obtaining the current concurrency count and interface performance of the target interface in the previous time period, the method further includes: The target interface is obtained by intercepting the server's interface using aspect-oriented programming.
9. The method according to any one of claims 1-5, characterized in that, The preset database is a remote dictionary service.
10. An interface request processing apparatus, characterized in that, include: The acquisition module is used to acquire the current concurrency of the target interface and the interface performance in the previous time period. The current concurrency represents the number of interface requests in the current time period that are in a concurrent state. The interface performance in the previous time period is determined based on the total time taken for the target interface to complete the interface requests and the total number of times the target interface completed the interface requests in the previous time period. The update module is used to update the current concurrency threshold cached in the preset database according to the preset configuration data and the interface performance, so as to obtain the updated current concurrency threshold. The processing module is used to perform rate limiting processing on the interface requests of the target interface if the current concurrency is greater than or equal to the updated current concurrency threshold. The preset configuration data includes a first preset threshold and a second preset threshold. The first preset threshold is a performance threshold for interfaces with increasing concurrency, and the second preset threshold is a performance threshold for interfaces with decreasing concurrency. The update module is used to: If the interface performance is less than or equal to the first preset threshold, the current concurrency threshold cached in the preset database is increased to obtain the updated current concurrency threshold. If the interface performance is greater than the first preset threshold and also greater than the second preset threshold, then the current concurrency threshold cached in the preset database is lowered to obtain the updated current concurrency threshold.
11. A server, characterized in that, include: Memory and at least one processor; The memory stores computer-executed instructions; The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the method as described in any one of claims 1-9.
12. A server cluster, characterized in that, Includes at least one server as described in claim 11.
13. A readable storage medium having a computer program stored thereon, characterized in that, The readable storage medium stores computer instructions that, when executed by a processor, are used to implement the method as described in any one of claims 1-9.
14. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method described in any one of claims 1-9.