A service system adaptive throttling method and device

Abstract

The application discloses a kind of service system self-adapting flow limiting method and device, it is related to computer technical field.The specific embodiment of the method includes: identifying each external service from service system, according to the traffic entering service system, generate the call relationship between each external service;According to call relationship, generate service dependency topology;Each physical node associated with external service is obtained;Calculate the proportion of traffic of external service on each physical node;According to the proportion of traffic and service dependency topology, self-adaptively adjust the flow limiting value of each external service and the flow limiting value of each physical node associated with external service, to limit the flow of external service and physical node according to the flow limiting value of external service and the flow limiting value of physical node.The embodiment can realize the self-adapting flow limiting of service system by service dependency topology and the mapping relationship between physical node and external service, improve the flow limiting effect, improve the availability and stability of service system.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to an adaptive rate limiting method and apparatus for a business system. Background Technology

[0002] Distributed systems are generally composed of multiple applications, each deployed in a distributed cluster. The clusters are often isolated into multiple groups, making deployment complex. They interact with external systems through various technology stacks. Therefore, building and maintaining distributed systems requires ensuring the stability and reliability of the system when facing high concurrency requests. Rate limiting is one of the key technologies to protect the system from overload.

[0003] In related technologies, there are two main ways to set rate limits: one is based on the adaptive rate limiting capability provided by middleware, and the other is based on rate limiting settings estimated from historical monitoring data. However, the above two methods are limited to single-point rate limiting and lack the ability to control the overall situation, which affects the availability and stability of the overall service. Moreover, single-point rate limiting is prone to cascading failures and back pressure, and is not suitable for hybrid service deployments. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide an adaptive rate limiting method and apparatus for a business system. By mapping the service dependency topology and physical nodes to external services, adaptive rate limiting of the business system can be achieved, thereby improving the rate limiting effect and enhancing the availability and stability of the business system.

[0005] To achieve the above objectives, according to one aspect of the present invention, an adaptive rate limiting method for a business system is provided, comprising:

[0006] Identify each external service from the business system, and generate the call relationship between the various external services based on the traffic entering the business system;

[0007] Generate a service dependency topology based on the calling relationships between the various external services;

[0008] For each external service, obtain the physical nodes associated with the external service; calculate the traffic percentage of the external service on each physical node;

[0009] Based on the traffic share of the external service on each physical node and the service dependency topology, the rate limiting value of each external service and the rate limiting value of each physical node associated with the external service are adaptively adjusted; rate limiting is applied to each external service and each physical node based on the rate limiting value of each external service and the rate limiting value of each physical node associated with the external service.

[0010] Optionally, various external services can be identified from the business system, including:

[0011] Obtain the code of the boundary application of the business system, and perform static code parsing on the code of the boundary application.

[0012] Identify the external services that meet the preset rules from the code of the boundary application.

[0013] Optionally, a service dependency topology is generated based on each external service and the call relationships between them, including:

[0014] Initialize the graph structure, creating a vertex for each external service in the graph structure;

[0015] For each external service, obtain other external services that have a calling relationship with the external service, and create directed edges between the external service and each other external service in the graph structure;

[0016] The service dependency topology is generated based on each vertex and the directed edges between them.

[0017] Optionally, obtaining each physical node associated with the external service includes:

[0018] For each external service, obtain the mapping relationship between the external service and the preset tag;

[0019] Acquire all traffic entering the business system within a preset time period;

[0020] For each traffic flow, if the traffic flows to the external service, the preset tag is added to the traffic flow; if the traffic flows through any physical node, the preset tag is added to any physical node.

[0021] Each physical node with the preset tag is designated as a physical node associated with the external service.

[0022] Optionally, the method further includes: when the traffic passes through any physical node, adding the traffic accumulation number corresponding to the preset tag to any physical node;

[0023] Calculate the traffic percentage of the external service on each physical node, including:

[0024] Within each collection period, the total traffic passing through the external service and the cumulative traffic corresponding to the preset marker on each physical node are obtained; the total traffic and the cumulative traffic corresponding to the preset marker on each physical node are normalized, and the candidate traffic ratio on each physical node within the collection period is calculated;

[0025] The largest of the multiple candidate traffic percentages on each physical node within multiple collection time periods is taken as the traffic percentage of the external service on that physical node.

[0026] Optionally, the service dependency topology includes multiple service relationship graphs, each consisting of one or more external services; based on the traffic share of each external service on each physical node and the service dependency topology, the rate limiting value of each external service and each physical node associated with the external service is adaptively adjusted, including:

[0027] For each external service that serves as an entry point in the service relationship graph, the rate limit value of each other external service in the service relationship graph is calculated based on the rate limit value of the external service and the service relationship graph.

[0028] For each external service, the rate limit value of the external service on each associated physical node is calculated based on the rate limit value of the external service and the traffic ratio of the external service on each associated physical node.

[0029] Optionally, the method further includes: for any physical node, adjusting the rate limiting value on the any physical node when a traffic change of the any physical node is detected;

[0030] Obtain the external service corresponding to any physical node as the entry point, adjust the rate limit value of the external service as the entry point based on the traffic ratio of the external service as the entry point on any physical node and the rate limit value of any physical node, and adjust the rate limit value of the downstream external service corresponding to the external service as the entry point based on the service dependency topology.

[0031] According to another aspect of the present invention, an adaptive rate limiting device for a business system is provided, comprising:

[0032] The acquisition module identifies various external services from the business system and generates the call relationships between these external services based on the traffic entering the business system.

[0033] The generation module generates a service dependency topology based on the call relationships between the various external services;

[0034] The calculation module, for each external service, obtains each physical node associated with the external service; and calculates the traffic percentage of the external service on each physical node.

[0035] The adjustment module adaptively adjusts the rate limiting value of each external service and the rate limiting value of each physical node associated with the external service based on the traffic ratio of the external service on each physical node and the service dependency topology; and applies rate limiting to each external service and each physical node based on the rate limiting value of each external service and the rate limiting value of each physical node associated with the external service.

[0036] According to another aspect of the present invention, an electronic device is provided, comprising:

[0037] One or more processors;

[0038] Storage device for storing one or more programs.

[0039] When the one or more programs are executed by the one or more processors, the one or more processors implement the adaptive rate limiting method for business systems provided by the present invention.

[0040] According to another aspect of the present invention, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the adaptive rate limiting method for a business system provided by the present invention.

[0041] An embodiment of the above invention has the following advantages or beneficial effects: The adaptive rate limiting method for business systems provided by this invention first identifies various external services in the business system, then generates call relationships between these external services based on the traffic entering the business system, and generates a service dependency topology based on these call relationships. Then, based on the service dependency topology, it obtains each physical node associated with each external service and calculates the traffic percentage of each external service on each physical node. Finally, based on the traffic percentage of each external service on each physical node and the service dependency topology, it adaptively adjusts the rate limiting value of each external service and the rate limiting value of each physical node associated with the external service. This method generates a service dependency topology between various external services and generates a physical node topology based on the service dependency topology, constructing a mapping relationship and traffic percentage relationship between external services and physical nodes, thereby achieving global rate limiting control, realizing adaptive rate limiting for the business system, improving the rate limiting effect, and enhancing the availability and stability of the business system.

[0042] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0043] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein:

[0044] Figure 1This is a schematic diagram of the main process of an adaptive rate limiting method for a business system according to an embodiment of the present invention;

[0045] Figure 2 This is a schematic diagram of a method for constructing a mapping relationship between physical nodes and external services based on traffic coloring according to an embodiment of the present invention;

[0046] Figure 3 This is a process based on determining the current limiting value of a DAG node according to an embodiment of the present invention;

[0047] Figure 4 This is a schematic diagram illustrating the process of calculating the traffic percentage of external services on physical nodes according to an embodiment of the present invention;

[0048] Figure 5 This is a schematic diagram of a fault impact scenario according to an embodiment of the present invention;

[0049] Figure 6 This is a schematic diagram of a DAG out-of-order scenario according to an embodiment of the present invention;

[0050] Figure 7 This is a schematic diagram of the main modules of the adaptive rate limiting device of the business system according to an embodiment of the present invention;

[0051] Figure 8 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied;

[0052] Figure 9 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers of the present invention. Detailed Implementation

[0053] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0054] For business systems designed with a distributed architecture, the system consists of multiple applications connected through message middleware. Each application is deployed in a distributed cluster, with the clusters grouped and isolated. Interaction with external systems occurs through various technology stacks such as MQ, RPC, and HTTP. This system is characterized by multiple applications, multiple technology stacks, complex interactions, and complex deployment. Building and maintaining a distributed system requires ensuring its stability and reliability under high concurrency requests. Rate limiting is one of the key technologies to protect the system from overload. There are two common methods for setting rate limits: one is based on the adaptive rate limiting capabilities provided by middleware. Taking RPC frameworks as an example, mature RPC frameworks provide adaptive rate limiting mechanisms. For instance, a service framework's adaptive rate limiting mechanism is based on concurrency. The adaptive rate limiting algorithm automatically assesses the performance bottleneck of remote service entities and adjusts the concurrency based on the carrying capacity of the remote service entities to achieve adaptive rate limiting. The other method is based on rate limiting settings estimated using historical monitoring data combined with statistical formulas: The system uses embedded APM (Application Performance Management) tools to statistically analyze the traffic of each core node, and sets the rate limit value by multiplying the historical peak traffic value by a prediction coefficient.

[0055] However, the first approach is designed for a specific technical framework and lacks the ability to control heterogeneous systems. Distributed systems often contain components from different technology stacks, such as different databases, message queues, and application servers. This limitation not only reduces the applicability of rate limiting strategies but also makes it impossible to form a unified traffic control strategy in multi-technology stack environments, increasing the complexity of system management. The second approach is based on historical statistical data estimation, which has poor accuracy and cannot support the setting of new features; it is also a semi-automatic design and cannot respond to system changes in real time. Furthermore, both approaches are unsatisfactory in terms of rate limiting effectiveness and resource utilization. Rate limiting capabilities are limited to a single point and lack global control capabilities: from a service perspective, distributed systems are usually composed of multiple nodes, and traffic may enter the system through different paths. If each node performs rate limiting independently, it is impossible to guarantee system-wide traffic control, which may cause some nodes to become bottlenecks due to traffic overload, affecting the overall availability and stability of the service. At the same time, in distributed systems, calls between services often form complex dependency chains; rate limiting at a single point can easily introduce cascading failures and back pressure. From a deployment perspective, each node in a distributed system (which can be understood as a server) hosts multiple services, and these services share server resources. Rate limiting mechanisms designed for individual services are not suitable for the current state of hybrid service deployments. In addition, rate limiting settings only aim to protect the system from sudden traffic surges, without considering the matching of target settings with resources, failing to identify resource bottlenecks and service capacity limitations, leading to resource waste.

[0056] Based on the above-mentioned technical problems, this invention provides an adaptive rate limiting method for business systems, which can globally control the rate limiting values ​​of each physical node in the business system, ensuring the availability and stability of the system when facing high concurrency requests. It is an adaptive and comprehensive rate limiting method that supports heterogeneous composition, global perspective management and is compatible with resource optimization.

[0057] Figure 1 This is a schematic diagram of the main flow of an adaptive rate limiting method for a business system according to an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes the following steps:

[0058] Step S101: Identify each external service from the business system, and generate the call relationship between each external service based on the traffic entering the business system;

[0059] Step S102: Generate a service dependency topology based on the call relationships between various external services;

[0060] Step S103: For each external service, obtain the physical nodes associated with the external service; calculate the traffic percentage of the external service on each physical node;

[0061] Step S104: Based on the traffic share of external services on each physical node and the service dependency topology, adaptively adjust the rate limiting value of each external service and the rate limiting value of each physical node associated with the external service; apply rate limiting to each external service and each physical node based on the rate limiting value of each external service and the rate limiting value of each physical node associated with the external service.

[0062] In this embodiment of the invention, the business system can be a distributed system, such as a fulfillment routing system in a logistics system. The system internally consists of multiple applications, including a routing gateway, routing implementation, routing calculation, and routing scheduling. These applications are interconnected via message queues and interact with external systems such as the pickup system, sorting system, and warehousing system, as well as other logistics systems, through various technology stacks such as message queues, RPC, and HTTP. The interaction entry points are primarily the inbound and outbound systems, such as the routing gateway and routing scheduling. Each application is deployed in a distributed cluster, with multiple isolated groups within the cluster, exhibiting characteristics of multiple applications, multiple technology stacks, complex interactions, and complex deployment.

[0063] In this embodiment of the invention, since the system code spans multiple applications, services, and is used in multiple service scenarios, static code analysis technology can be used to identify various external services from the business system. These external services are those that interact with external systems within the business system. Specifically, identifying various external services from the business system includes: obtaining the code of the boundary applications of the business system; performing static code analysis on the boundary application code; and identifying various external services that meet preset rules from the boundary application code.

[0064] In this embodiment of the invention, external services refer to services that interact with external systems. When identifying various external services from the business system, the language and framework used by the business system are first determined. For example, a logistics fulfillment routing system is implemented using Java code and the Spring framework, providing external services through MQ, RPC, REST, etc. Then, preset rules for identifying external services are defined. First, the code of the boundary application is obtained from the business system. The boundary application is the application in the business system that interacts with external systems, such as the routing gateway application. The routing gateway is designed based on the scenario of external services, with few services and clear dependencies, using the gateway application as the entry point to shield the complexity of internal applications. Then, static code analysis is performed on the code of the boundary application. The static code analysis tool is embedded in the compiler compilation process, and the code is traversed and compiled. The code of the boundary application is matched with the preset rules to identify external services that meet the preset rules. The preset rules include the service type, service annotation description, fully qualified class name, etc. If any service successfully matches the preset rules, it is an external service that meets the preset rules. The preset rules are set according to different service types.

[0065] Taking a logistics fulfillment routing system as an example, a Spring framework Java application REST service needs to find classes annotated with `@RestController` and methods annotated with `@RequestMapping`; a message queue (MQ) needs to find methods that implement `MessageListener` or... <mq:listener>The configured class; for RPC, you need to look for the @Service annotation or <jsf:provider>The configuration class. For Java applications, the static code analysis tool can be configured with plugins such as Checkstyle, PMD, or SpotBugs, or a custom tool like ANTLR (a tool for generating language parsers, compilers, and interpreters) can be used.

[0066] In this embodiment of the invention, a list of external services, i.e., each external service, can be obtained from the business system through static code parsing. Then, the call relationships between these external services need to be determined to generate a service dependency topology based on the individual external services and their call relationships. The call relationships between the external services can be generated based on the traffic entering the business system. Specifically, application performance management (APM) tools can be used to automatically track and record the traffic entering the business system, and the call relationships between external services can be obtained through dynamic analysis. For example, the call relationships between external services can be determined based on the time sequence of external services accessed by the same traffic, generating a service call chain.

[0067] In this embodiment of the invention, generating a service dependency topology based on each external service and the calling relationships between them includes:

[0068] Initialize the graph structure, and create a vertex for each external service in the graph structure;

[0069] For each external service, obtain other external services that have a calling relationship with the external service, and create directed edges between the external service and each other external service in the graph structure;

[0070] Generate service dependency topology based on each vertex and the directed edges between them.

[0071] In this embodiment of the invention, a service dependency topology is generated based on each external service and the calling relationships between them. The service dependency topology can be a set of multiple DAGs (Directed Acyclic Graphs). Specifically, each external service is obtained, i.e., a list or set containing each external service, such as ["ServiceA", "ServiceB", "ServiceC",...]. Then, the calling relationships between each external service are obtained, i.e., an external service call list. This call list includes multiple elements, i.e., multiple call chains. Each element represents an ordered pair of call chains, such as [("ServiceA", "ServiceB"), ("ServiceB", "ServiceC"), ...].

[0072] Then, the graph structure is initialized, i.e., a data structure representing the graph is created. Adjacency lists or adjacency matrices can be used; for DAGs, adjacency lists are more space-efficient. Each external service is traversed, and a vertex is created for each service. Based on each call chain in the call chain, other external services with call relationships to each service are obtained, and then directed edges are used to connect the two vertices according to the call relationships. Multiple DAGs can be generated based on each vertex and directed edge. Each DAG can include one or more external services, forming a DAG set. This set of DAGs, consisting of the external services and their call relationships, constitutes the generated service dependency topology. Generating service dependency topologies can solve the single-point constraint problem in rate limiting schemes in related technologies, providing a basis for global management. Each DAG is a service relationship graph, and each service relationship graph includes a DAG node as the entry point, i.e., the external service that serves as the entry point.

[0073] During the generation of a DAG, a depth-first search (DFS) method can be used to detect cycles. If a cycle is detected, it can be eliminated by deleting directed edges or merging nodes.

[0074] In this embodiment of the invention, obtaining each physical node associated with the external service includes:

[0075] For each external service, obtain the mapping relationship between the external service and the preset tag;

[0076] Acquire all traffic entering the business system within a preset time period;

[0077] For each traffic item, when the traffic accesses an external service, a preset tag is added to the traffic; when the traffic passes through any physical node, a preset tag is added to any physical node.

[0078] Each physical node with a preset label is designated as a physical node associated with external services.

[0079] In this embodiment of the invention, after generating the service dependency topology, a physical node topology is generated based on the service dependency topology. This is because the service dependency topology is a service network topology based on a logical perspective, while the specific implementation process of the service is scattered throughout the complex physical deployment. Therefore, it is necessary to establish a mapping relationship between the service dependency topology and physical nodes. Due to the complexity of the physical deployment and service calls of the business system, the implementation of external services is supported by a combination of multiple internal services. These internal services are fine-grained, large-scale, cross-application, and cross-node, and they are reused to varying degrees. Therefore, the complexity of physical deployment and internal service calls can be shielded to generate a physical node topology.

[0080] In this embodiment of the invention, when generating the physical node topology, each physical node can be a server, and each physical node can include multiple internal services. All physical nodes can be homogeneous, maintaining only the association with external services. The access list and access volume of external services for each physical node are counted in real time to obtain the physical node topology. When generating the physical node topology, for each external service, the physical nodes associated with the external service are first obtained. Specifically, each DAG node in the service dependency topology is obtained, i.e., each external service. For each external service, the mapping relationship between the external service and the preset marker is obtained. That is, before this, the mapping relationship between the external service and the preset marker needs to be pre-configured. Different colors can be used to distinguish different external services, or other methods can be used to distinguish different external services. For example, the mapping relationship between the external service and the preset marker can be service coloring metadata, including the mapping relationship between each external service and the corresponding color marker, such as ("ServiceA", "red"), ("ServiceB", "yellow"), ....

[0081] A proxy layer is embedded in each external service. This proxy layer colors the traffic when it accesses the external service. After coloring, all physical nodes traversed by the traffic carry the coloring information. A colored physical node indicates that it contains an internal service associated with the external service. Specifically, each traffic entering the business system within a preset time period is acquired. For each traffic, if it accesses the external service, a preset tag is added to the traffic, i.e., the traffic is colored. Then, the physical nodes traversed by the colored traffic are tracked, and a preset tag is added to each physical node, i.e., the physical nodes are colored. All physical nodes with preset tags are considered as physical nodes associated with the external service.

[0082] Figure 2 This is a schematic diagram illustrating a process for constructing a mapping relationship between physical nodes and external services based on traffic coloring, according to an embodiment of the present invention. The gateway includes a service dependency topology composed of three Directed Acyclic Graphs (DAGs), each consisting of one, two, and four external services, respectively. First, each external service in the gateway is colored. When traffic enters, hierarchical statistics can be performed based on the number of DAGs; the three DAGs can be divided into three layers. For each layer, when traffic passes through that DAG, it is colored with the color corresponding to the external service at the DAG's entry point. Then, each physical node that the traffic passes through is colored, thus obtaining the mapping relationship between each external service in the DAG and the colored physical node. This yields red, green, and blue hierarchical statistical results, respectively. The hierarchical statistical results are then fused to obtain the mapping relationship between each physical node and an external service. Some physical nodes can correspond to multiple external services, thus obtaining the physical node associated with each external service.

[0083] In this embodiment of the invention, the method further includes: when traffic passes through any physical node, adding a traffic accumulation number corresponding to a preset marker to any physical node;

[0084] Calculate the traffic percentage of external services on each physical node, including:

[0085] Within each collection period, the total traffic that has been served externally and the cumulative number of traffic corresponding to the preset marker on each physical node are obtained; the total traffic and the cumulative number of traffic corresponding to the preset marker on each physical node are normalized, and the proportion of candidate traffic on each physical node within the collection period is calculated.

[0086] The largest percentage of candidate traffic on each physical node within multiple collection time periods is taken as the percentage of traffic for external services on that physical node.

[0087] In this embodiment of the invention, adding a preset marker to a physical node can be achieved by embedding coloring information in the physical node's service APM to count the cumulative number of traffic of each color passing through the physical node. For example, each time traffic passes through the physical node, it can be counted as external service, color, and access increment, such as ("ServiceA", "red", +1). Then, each physical node aggregates all traffic and performs color fusion processing to obtain the cumulative number of traffic corresponding to the preset marker added to the physical node, i.e., the access count of different colors, such as ("red", 100). This results in a mapping list of each external service (i.e., each preset marker) and the cumulative number of traffic for each physical node within a preset time period.

[0088] In this embodiment of the invention, for each collection time period, the total traffic passing through the external service and the cumulative traffic of each preset marker on each physical node, i.e., each external service, are obtained. Then, normalization processing is performed to normalize the total traffic to 1. The cumulative traffic of each external service is divided by the total traffic to obtain the normalized result of each external service, i.e., the candidate traffic ratio of the external service on each physical node. Then, multiple collection time periods are iterated to update the candidate traffic ratio. Specifically, the Max function can be used for updating, that is, the largest candidate traffic ratio among the multiple candidate traffic ratios of the external service on the physical node within multiple collection time periods is taken as the traffic ratio of the external service on the physical node.

[0089] In this embodiment of the invention, the service dependency topology includes multiple service relationship graphs, each consisting of one or more external services; based on the traffic share of the external service on each physical node and the service dependency topology, the rate limiting values ​​of each external service and each physical node associated with the external service are adaptively adjusted, including:

[0090] For each external service that serves as an entry point in the service relationship graph, calculate the rate limit value for each other external service in the service relationship graph based on the rate limit value of the external service and the service relationship graph.

[0091] For each external service, the rate limit value for the external service on each associated physical node is calculated based on the rate limit value of the external service and the traffic proportion of the external service on each associated physical node.

[0092] In this embodiment of the invention, after obtaining the traffic percentage of the external service on each physical node, the global rate limit value can be obtained by combining the service dependency topology, that is, the rate limit value of each external service and the rate limit value of each physical node associated with the external service. Specifically, the external service serving as the entry point can be obtained from the service dependency topology. By setting the rate limit value for the external service serving as the entry point, the rate limit value for each external service can be obtained based on the service dependency topology. For example, for a service relationship graph (DAG) containing one external service, the rate limit value for that external service can be set. For a service relationship graph containing multiple external services, the traffic of each external service in the service dependency topology is statistically analyzed over multiple collection periods. Then, using the external service serving as the entry point in the service relationship graph as the benchmark, the traffic of each other external service in the service relationship graph is normalized to obtain the traffic ratio relationship between each external service. The traffic ratio relationship is iteratively updated over multiple collection periods, using the Max function during the update process, and each external service retains the largest traffic ratio in each update. After obtaining the traffic ratio relationship of each external service in the service relationship graph, by setting the rate limit value for the external service serving as the entry point in the service relationship graph, the rate limit value for each other external service can be obtained based on the traffic ratio relationship, and thus the rate limit value for each external service can be obtained. The rate limiting values ​​for specific external services used as entry points can be strongly correlated with specific business forecasts. Taking a fulfillment routing system as an example, the peak volume of waybill data during peak hours will reach 200,000 / s. This 200,000 / s is a business forecast, and the system's resource preparation needs to meet this standard. When the rate limiting value of a certain external service is updated, the rate limiting values ​​of other external services with which that service has a calling relationship will also be dynamically updated in real time based on the traffic ratio.

[0093] In this embodiment of the invention, based on the rate limiting value of each external service and the traffic ratio of the external service on each associated physical node, the rate limiting value of the external service on each associated physical node can be obtained. Then, rate limiting is applied to each external service and each physical node based on the rate limiting value of each external service and the rate limiting value of each physical node associated with the external service, so as to control the traffic of each external service and each physical node.

[0094] Figure 3 This diagram illustrates the calculation of rate limiting values ​​for DAG nodes according to an embodiment of the present invention. The diagram shows the calling relationships between the various DAG nodes. During the initialization phase, a collection period is set, and the input traffic of each entry node serving as an external service is acquired during each collection period (e.g., 50). Then, based on the DAG graph, the traffic of the other DAG nodes serving the external service of the entry node is statistically analyzed. Next, the input traffic of the entry node's external service is used as a benchmark to normalize the traffic of the other DAG nodes, resulting in the normalized proportion of each DAG node. Through iterative updates of the collection period, the proportions of each DAG node are updated using the Max function, obtaining the proportional relationship between each DAG node and the entry node. By setting the rate limiting value of the entry node, the rate limiting values ​​of the other DAG nodes can be obtained based on the proportional relationship.

[0095] Figure 4 This is a flowchart illustrating a method for calculating the rate limit value of an external service on physical nodes according to an embodiment of the present invention. Taking any external service (such as an external service acting as an entry point) as an example, the process begins with an initialization phase. A data collection period is set, and the input traffic of the external service acting as the entry point is counted (50). Then, the cumulative traffic of this external service on each associated physical node is counted, as shown in red in the figure (10, 8, 12, 6, and 14). Using the input traffic of the external service at the entry point as a benchmark, the cumulative traffic on each physical node is normalized to obtain the traffic percentage on each physical node, as shown in red in the figure (0.2, 0.16, 0.24, 0.12, and 0.28). Through iterative updates during the data collection period, the traffic percentage is continuously updated using the update function Max. The larger traffic percentage is taken in each update, thus obtaining the traffic percentage of the external service on each physical node. By setting the rate limit value of the entry point external service and combining it with the traffic percentage, the rate limit value of the external service on each associated physical node can be obtained.

[0096] In this embodiment of the invention, the method further includes: for any physical node, when a traffic change of any physical node is detected, adjusting the rate limiting value on any physical node; obtaining the external service corresponding to any physical node as the entry point, adjusting the rate limiting value of the external service as the entry point based on the traffic ratio of the external service as the entry point on any physical node and the rate limiting value of any physical node, and adjusting the rate limiting value of the downstream external service corresponding to the external service as the entry point based on the service dependency topology.

[0097] In this embodiment of the invention, each physical node can act as an independent fault-aware individual. When any physical node fails, i.e., when the traffic of that physical node changes, the traffic change is acquired, and the rate limiting value of that physical node is adjusted in real time based on the traffic change. For example, the rate limiting value corresponding to each external service on that physical node can be adjusted, and then traced back to each external service that serves as an entry point on that physical node. Based on the traffic ratio and the rate limiting value of that physical node, the rate limiting value of the external service serving as an entry point is adjusted. Thus, based on the traffic ratio and the traffic proportion, the rate limiting value of each external service and each external service associated with each physical node can be obtained, realizing global control of rate limiting capabilities and achieving adaptive rate limiting.

[0098] After setting the global rate limit, the system load can be tested to reach the rate limit threshold. Then, the rationality of the resource setting can be analyzed based on the test results, and resource configuration can be optimized to avoid resource waste.

[0099] In this embodiment of the invention, the above-mentioned fault handling process can be embedded within each physical node or can rely on a separate rate limiting server as a fallback. The rate limiting server maintains a heartbeat connection with each physical node. In the event of a physical node failure (inability to perform computation), the rate limiting server takes over from the physical node and adjusts the rate limiting of the entry node. During each heartbeat synchronization, the physical node reports rate limiting-related metadata information to the rate limiting server, supporting global rate limiting optimization by the rate limiting server.

[0100] When the processing order of DAG nodes in a service-dependent topology is out of order, the service is invalid due to the out-of-order calculation. Therefore, the calculation needs to be reordered after the service of the preceding node is triggered, which introduces additional traffic. Figure 5 This diagram illustrates a DAG out-of-order issue causing a failure. For example, in a fulfillment routing system, there might be a DAG dependency: {Waybill Creation -> Order Placement -> Warehouse Order Acceptance -> Order Packaging}. Ideally, these four service requests arrive in the system sequentially, without out-of-order issues. However, in an out-of-order scenario, the order might be Order Placement - Warehouse Order Acceptance - Waybill Creation - Order Packaging. By detecting the out-of-order issue and rearranging the order, Order Placement - Warehouse Order Acceptance - Waybill Creation can generate replay traffic, impacting system stability. In this case, a DAG dependency-aware synchronous rate limiting approach can be adopted. When adjusting the rate limit value for directly mapped DAG nodes due to a physical node failure, other DAG nodes are traversed, and the rate limit values ​​for affected downstream links (indirectly affected nodes) are adjusted synchronously. When setting rate limits, it's necessary to ensure that larger rate limit values ​​override smaller ones.

[0101] Figure 6 This is a flowchart illustrating an adaptive rate limiting method for a business system according to an embodiment of the present invention. Based on the engineering code of the boundary application of the business system, various external services are obtained through static code parsing. The call relationships between these external services are obtained based on dynamic traffic analysis. A service dependency topology is generated based on the call relationships between these external services and external accesses. A physical node topology is generated based on the service dependency topology (i.e., a service topology forest, a set of multiple Directed Acyclic Graphs), and the physical node topology (i.e., the mapping relationship between physical nodes and external services). Based on the target traffic setting, and combining the service topology forest and the mapping relationship between physical nodes and external services, a rate limiting value set for each external service and each physical node is obtained through forward calculation. In fault response scenarios, adaptive rate limiting can be achieved by combining the service topology forest and the mapping relationship between physical nodes and external services, updating the rate limiting value set for each physical node.

[0102] The adaptive rate limiting method for business systems provided in this invention first identifies various external services from the business system, then generates call relationships between these external services based on the traffic entering the business system, and generates a service dependency topology based on these call relationships. Then, based on the service dependency topology, it obtains each physical node associated with each external service and calculates the traffic percentage of each external service on each physical node. Finally, based on the traffic percentage of each external service on each physical node and the service dependency topology, it adaptively adjusts the rate limiting value for each external service and the rate limiting value for each physical node associated with the external service. This method generates service dependency topologies between various external services and then generates physical node topologies based on these topologies. It establishes mapping relationships and traffic proportions between external services and physical nodes, enabling global rate limiting control and adaptive rate limiting for the business system. This improves rate limiting effectiveness and enhances the availability and stability of the business system. This method is suitable for complex business system designs and deployments. It generates service dependency topologies based on a combination of static code analysis and dynamic traffic analysis, and constructs mapping relationships between physical nodes and external services based on these topologies and traffic coloring. It sets and automatically generates rate limiting value sets for each physical node in the entire business system under two scenarios: target setting and fault response, thus improving adaptive rate limiting capabilities. Furthermore, this method enables reasonable resource distribution, avoiding resource waste introduced by rate limiting. It supports heterogeneous architectures and can be applied to rate limiting in distributed systems with multiple technology stacks.

[0103] According to another aspect of the embodiments of the present invention, such as Figure 7 As shown, an adaptive rate limiting device 700 for a business system is provided, comprising:

[0104] The acquisition module 701 identifies various external services from the business system and generates the call relationships between these external services based on the traffic entering the business system.

[0105] Module 702 generates service dependency topology based on the call relationships between various external services;

[0106] The calculation module 703, for each external service, obtains each physical node associated with the external service; and calculates the traffic share of the external service on each physical node.

[0107] The adjustment module 704 adaptively adjusts the rate limiting value of each external service and the rate limiting value of each physical node associated with the external service based on the traffic ratio of the external service on each physical node and the service dependency topology; and applies rate limiting to each external service and each physical node based on the rate limiting value of each external service and the rate limiting value of each physical node associated with the external service.

[0108] In this embodiment of the invention, the acquisition module 701 is further configured to: acquire the code of the boundary application of the business system, perform static code parsing on the code of the boundary application, and identify various external services that meet preset rules from the code of the boundary application.

[0109] In this embodiment of the invention, the generation module 702 is further configured to: initialize a graph structure, create a vertex for each external service in the graph structure; for each external service, obtain other external services that have a calling relationship with the external service, create directed edges between the external service and each other external service in the graph structure; and generate a service dependency topology based on each vertex and the directed edges between each vertex.

[0110] In this embodiment of the invention, the calculation module 703 is further configured to: for each external service, obtain the mapping relationship between the external service and the preset tag; obtain each traffic entering the business system within a preset time; for each traffic, add a preset tag to the traffic when the traffic accesses the external service; add a preset tag to any physical node when the traffic passes through any physical node; and use each physical node with the preset tag as each physical node associated with the external service.

[0111] In this embodiment of the invention, the calculation module 703 is further configured to: add a cumulative number of traffic corresponding to a preset marker to any physical node when traffic passes through any physical node;

[0112] The calculation module 703 is further configured to: obtain the total traffic that has been used for external services and the cumulative number of traffic corresponding to the preset mark on each physical node within each collection time period; normalize the total traffic and the cumulative number of traffic corresponding to the preset mark on each physical node to calculate the candidate traffic ratio on each physical node within the collection time period; and take the largest of the multiple candidate traffic ratios on each physical node within multiple collection time periods as the traffic ratio of external services on that physical node.

[0113] In this embodiment of the invention, the service dependency topology includes multiple service relationship graphs, each of which consists of one or more external services; the adjustment module 704 is further configured to: for each external service that serves as an entry point in the service relationship graph, calculate the rate limit value of each other external service in the service relationship graph based on the rate limit value of the external service and the service relationship graph; for each external service, calculate the rate limit value of the external service on each associated physical node based on the rate limit value of the external service and the traffic ratio of the external service on each associated physical node.

[0114] In this embodiment of the invention, the adjustment module 704 is further configured to: for any physical node, when a change in the traffic of any physical node is detected, adjust the rate limiting value on any physical node; obtain the external service corresponding to any physical node as the entry point, adjust the rate limiting value of the external service as the entry point according to the traffic ratio of the external service as the entry point on any physical node and the rate limiting value of any physical node, and adjust the rate limiting value of the downstream external service corresponding to the external service as the entry point according to the service dependency topology.

[0115] According to another aspect of the present invention, an electronic device is provided, comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the adaptive rate limiting method for business systems provided by the present invention.

[0116] According to another aspect of the present invention, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the adaptive rate limiting method for a business system provided by the present invention.

[0117] Figure 8 An exemplary system architecture 800 is shown that can be applied to the adaptive rate limiting method or device for business systems according to embodiments of the present invention.

[0118] like Figure 8 As shown, system architecture 800 may include terminal devices 801, 802, and 803, a network 804, and a server 805. Network 804 serves as the medium for providing communication links between terminal devices 801, 802, and 803 and server 805. Network 804 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0119] Users can use terminal devices 801, 802, and 803 to interact with server 805 via network 804 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 801, 802, and 803, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0120] Terminal devices 801, 802, and 803 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0121] Server 805 can be a server providing various services, such as a backend management server supporting shopping websites browsed by users using terminal devices 801, 802, and 803 (for example only). The backend management server can analyze and process data such as received product information query requests, and feed back the processing results (such as target push information and product information - for example only) to the terminal devices.

[0122] It should be noted that the adaptive rate limiting method for business systems provided in this embodiment of the invention is generally executed by server 805, and correspondingly, the adaptive rate limiting device for business systems is generally set in server 805.

[0123] It should be understood that Figure 8 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0124] The following is for reference. Figure 9 It shows a schematic diagram of the structure of a computer system 900 suitable for implementing a terminal device of the present invention. Figure 9 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0125] like Figure 9 As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 902 or programs loaded from storage section 908 into random access memory (RAM) 903. The RAM 903 also stores various programs and data required for the operation of the system 900. The CPU 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0126] The following components are connected to I / O interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to I / O interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 910 as needed so that computer programs read from it can be installed into storage section 908 as needed.

[0127] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 909, and / or installed from removable medium 911. When the computer program is executed by central processing unit (CPU) 901, it performs the functions defined above in the system of this invention.

[0128] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0129] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0130] The modules described in the embodiments of the present invention can be implemented in software or hardware. The described modules can also be housed in a processor; for example, a processor can be described as including an acquisition module, a generation module, a calculation module, and an adjustment module. The names of these modules do not necessarily limit the module itself; for example, the acquisition module can also be described as "a module that identifies various external services from the business system."

[0131] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs that, when executed by the device, cause the device to include: identifying various external services from a business system; generating call relationships between the various external services based on traffic entering the business system; generating a service dependency topology based on the call relationships between the various external services; for each external service, obtaining each physical node associated with the external service; calculating the traffic share of the external service on each physical node; adaptively adjusting the rate limiting value of each external service and the rate limiting value of each physical node associated with the external service based on the traffic share of the external service on each physical node and the service dependency topology; and applying rate limiting to each external service and each physical node based on the rate limiting value of each external service and the rate limiting value of each physical node associated with the external service.

[0132] According to the technical solution of the present invention, the adaptive rate limiting method for a business system provided by the present invention first identifies each external service from the business system, then generates the call relationship between each external service based on the traffic entering the business system, and generates a service dependency topology based on the call relationship between each service; then, based on the service dependency topology, obtains each physical node associated with each external service, and calculates the traffic proportion of the external service on each physical node; then, based on the traffic proportion of the external service on each physical node and the service dependency topology, adaptively adjusts the rate limiting value of each external service and the rate limiting value of each physical node associated with the external service. This method generates service dependency topologies between various external services and then generates physical node topologies based on these topologies. It establishes mapping relationships and traffic proportions between external services and physical nodes, enabling global rate limiting control and adaptive rate limiting for the business system. This improves rate limiting effectiveness and enhances the availability and stability of the business system. This method is suitable for complex business system designs and deployments. It generates service dependency topologies based on a combination of static code analysis and dynamic traffic analysis, and constructs mapping relationships between physical nodes and external services based on these topologies and traffic coloring. It sets and automatically generates rate limiting value sets for each physical node in the entire business system under two scenarios: target setting and fault response, thus improving adaptive rate limiting capabilities. Furthermore, this method enables reasonable resource distribution, avoiding resource waste introduced by rate limiting. It supports heterogeneous architectures and can be applied to rate limiting in distributed systems with multiple technology stacks.

[0133] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.< / jsf:provider> < / mq:listener>

Claims

1. An adaptive rate limiting method for a business system, characterized in that, include: Identify each external service from the business system, and generate the call relationship between the various external services based on the traffic entering the business system; Generate a service dependency topology based on the calling relationships between the various external services; For each external service, obtain the physical nodes associated with the external service; Calculate the traffic percentage of the external service on each physical node; Based on the traffic share of the external service on each physical node and the service dependency topology, the rate limiting value of each external service and the rate limiting value of each physical node associated with the external service are adaptively adjusted; rate limiting is applied to each external service and each physical node based on the rate limiting value of each external service and the rate limiting value of each physical node associated with the external service.

2. The method according to claim 1, characterized in that, Identify the various external services from the business system, including: Obtain the code of the boundary application of the business system, and perform static code parsing on the code of the boundary application. Identify the external services that meet the preset rules from the code of the boundary application.

3. The method according to claim 2, characterized in that, A service dependency topology is generated based on each external service and the call relationships between them, including: Initialize the graph structure, creating a vertex for each external service in the graph structure; For each external service, obtain other external services that have a calling relationship with the external service, and create directed edges between the external service and each other external service in the graph structure; The service dependency topology is generated based on each vertex and the directed edges between them.

4. The method according to claim 1, characterized in that, Obtain the physical nodes associated with the external service, including: For each external service, obtain the mapping relationship between the external service and the preset tag; Acquire all traffic entering the business system within a preset time period; For each traffic flow, if the traffic flows to the external service, the preset tag is added to the traffic flow; if the traffic flows through any physical node, the preset tag is added to any physical node. Each physical node with the preset tag is designated as a physical node associated with the external service.

5. The method according to claim 4, characterized in that, The method further includes: when the traffic passes through any physical node, adding the traffic accumulation number corresponding to the preset marker to any physical node; Calculate the traffic percentage of the external service on each physical node, including: Within each collection period, the total traffic passing through the external service and the cumulative traffic corresponding to the preset marker on each physical node are obtained; the total traffic and the cumulative traffic corresponding to the preset marker on each physical node are normalized, and the candidate traffic ratio on each physical node within the collection period is calculated; The largest of the multiple candidate traffic percentages on each physical node within multiple collection time periods is taken as the traffic percentage of the external service on that physical node.

6. The method according to claim 1, characterized in that, The service dependency topology includes multiple service relationship graphs, each consisting of one or more external services; based on the traffic share of each external service on each physical node and the service dependency topology, the rate limiting value of each external service and each physical node associated with the external service is adaptively adjusted, including: For each external service that serves as an entry point in the service relationship graph, the rate limit value of each other external service in the service relationship graph is calculated based on the rate limit value of the external service and the service relationship graph. For each external service, the rate limit value of the external service on each associated physical node is calculated based on the rate limit value of the external service and the traffic ratio of the external service on each associated physical node.

7. The method according to claim 1, characterized in that, The method further includes: For any physical node, if a change in the traffic of any physical node is detected, adjust the rate limit value on that physical node. Obtain the external service corresponding to any physical node as the entry point, adjust the rate limit value of the external service as the entry point based on the traffic ratio of the external service as the entry point on any physical node and the rate limit value of any physical node, and adjust the rate limit value of the downstream external service corresponding to the external service as the entry point based on the service dependency topology.

8. An adaptive rate limiting device for a business system, characterized in that, include: The acquisition module identifies various external services from the business system and generates the call relationships between these external services based on the traffic entering the business system. The generation module generates a service dependency topology based on the call relationships between the various external services; The computing module, for each external service, obtains the physical nodes associated with the external service; Calculate the traffic percentage of the external service on each physical node; The adjustment module adaptively adjusts the rate limiting value of each external service and the rate limiting value of each physical node associated with the external service based on the traffic ratio of the external service on each physical node and the service dependency topology; and applies rate limiting to each external service and each physical node based on the rate limiting value of each external service and the rate limiting value of each physical node associated with the external service.

9. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-7.

10. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

Images