Flow control system, method and apparatus

Abstract

The application provides a traffic control system, method, device, equipment, storage medium and computer program product; the system comprises: a traffic management service and at least one traffic control device corresponding to a service cluster; the traffic control device is used for sending a target traffic allocation quota acquisition request to the traffic management service when a target access request for the service cluster is received; the traffic management service is used for acquiring a total traffic allocation quota of the service cluster; in response to the acquisition request, the target traffic allocation quota of the traffic control device is determined based on the total traffic allocation quota and the number of historical access requests corresponding to each traffic control device, and the target traffic allocation quota is sent to the traffic control device; the traffic control device is also used for determining that the access operation corresponding to the target access request will be executed when the number of received target access requests does not reach the target traffic allocation quota; through the application, the efficiency and effect of traffic control can be improved.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to a flow control system, method, apparatus, device, storage medium, and computer program product. Background Technology

[0002] With the development of internet technology, internet-based business services are becoming increasingly popular. When a large number of users access these services simultaneously, the business service cluster providing these services may experience a surge in traffic. Related technologies address this by implementing traffic control for each service within the business service cluster based on its own hardware and software resource consumption thresholds (such as CPU, memory, and network card).

[0003] Business services typically require multiple services to work together. If one of these services limits traffic because it cannot support the current access traffic, it will lead to a waste of computing resources for other services, resulting in unreasonable and inefficient traffic control. At the same time, a surge in access traffic can also cause services to malfunction due to excessive instantaneous load, making the traffic control effect unsatisfactory. Summary of the Invention

[0004] This application provides a flow control system, method, apparatus, device, storage medium, and computer program product.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a traffic control system, including: a traffic management service, and at least one traffic control device corresponding to a service service cluster; wherein...

[0007] The traffic control device is used to send a target traffic allocation quota acquisition request to the traffic management service when it receives a target access request for the business service cluster. The target traffic allocation quota is used to indicate the maximum number of access requests that can be responded to within a unit of time.

[0008] The traffic management service is used to obtain the total traffic allocation limit of the business service cluster, and the total traffic allocation limit is used to indicate the number of access requests that the business service cluster can support in responding to.

[0009] In response to the acquisition request, based on the total traffic allocation quota and the number of historical access requests corresponding to each of the traffic control devices, the target traffic allocation quota of the traffic control device is determined, and the target traffic allocation quota is sent to the traffic control device.

[0010] The flow control device is further configured to determine that the access operation corresponding to the target access request will be executed when the number of received target access requests does not reach the target flow allocation quota.

[0011] In the above scheme, the traffic control device is further configured to obtain the access quality score corresponding to the business service cluster when the number of received target access requests does not reach the target traffic allocation quota;

[0012] When the access quality score is greater than the first quality score threshold, it is determined that the access operation corresponding to the target access request will be executed.

[0013] In the above scheme, the flow control device is also used to obtain access quality assessment information of the business service cluster, and the access quality assessment information includes at least one of the following: access response time and response error probability;

[0014] Based on the access quality assessment information, the access quality score corresponding to the business service cluster is determined.

[0015] This application also provides a traffic control method, applied to a traffic control system including a traffic management service and at least one traffic control device corresponding to a business service cluster; including:

[0016] When the traffic control device receives a target access request for the business service cluster, it sends a request to obtain the target traffic allocation quota to the traffic management service.

[0017] The acquisition request is used for the traffic management service to respond to the acquisition request, determine the target traffic allocation quota of the traffic control device based on the total traffic allocation quota of the business service cluster and the number of historical access requests corresponding to each traffic control device, and return it.

[0018] Wherein, the total traffic allocation quota is used to indicate the number of access requests that the business service cluster can support, and the target traffic allocation quota is used to indicate the maximum number of access requests that can be responded to within a unit of time.

[0019] Receive the target traffic allocation quota returned based on the acquisition request;

[0020] If the number of received target access requests does not reach the target traffic allocation quota, it is determined that the access operation corresponding to the target access request will be executed.

[0021] This application embodiment also provides a flow control device, applied to a flow control system including a flow management service and at least one flow control device corresponding to a service service cluster; including:

[0022] The sending module is used to send a request to obtain the target traffic allocation quota to the traffic management service when it receives a target access request for the business service cluster.

[0023] The acquisition request is used for the traffic management service to respond to the acquisition request, determine the target traffic allocation quota of the traffic control device based on the total traffic allocation quota of the business service cluster and the number of historical access requests corresponding to each traffic control device, and return it.

[0024] Wherein, the total traffic allocation quota is used to indicate the number of access requests that the business service cluster can support, and the target traffic allocation quota is used to indicate the maximum number of access requests that can be responded to within a unit of time.

[0025] The receiving module is configured to receive the target traffic allocation quota returned based on the acquisition request;

[0026] The determination module is used to determine that when the number of received target access requests does not reach the target traffic allocation quota, the access operation corresponding to the target access request will be executed.

[0027] This application also provides an electronic device, including:

[0028] Memory, used to store executable instructions;

[0029] The processor, when executing executable instructions stored in the memory, implements the flow control method provided in the embodiments of this application.

[0030] This application also provides a computer-readable storage medium storing executable instructions, which, when executed by a processor, implement the flow control method provided in this application.

[0031] This application also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the flow control method provided in this application.

[0032] The embodiments of this application have the following beneficial effects:

[0033] According to the embodiments of this application, the traffic management service reasonably allocates a target traffic allocation quota (indicating the maximum number of access requests allowed to be responded to within a unit of time) to the business service cluster based on the total traffic allocation quota of the business service cluster (used to indicate the number of access requests that the business service cluster can support). Thus, the traffic control device that controls access requests entering the business service cluster performs traffic control on the access requests entering the business service cluster according to the allocated target traffic allocation quota. That is, when the traffic control device determines that the number of received access requests has not reached the target traffic allocation quota, it determines that the access operation corresponding to the target access request will be executed, that is, the target access request is allowed to access the business service cluster.

[0034] In this way, by controlling the ingress traffic of the business service cluster, overall traffic control of the business service cluster can be achieved, improving the utilization rate of service computing resources and enhancing the rationality and efficiency of traffic control. At the same time, it reduces the occurrence of service anomalies due to excessive instantaneous load, thus improving the effectiveness of traffic control. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the architecture of the flow control system 100 provided in an embodiment of this application;

[0036] Figure 2 This is a schematic diagram of the structure of the electronic device 500 implementing the flow control method provided in the embodiments of this application;

[0037] Figure 3 This is a schematic diagram of the interaction process between various devices in the flow control system provided in the embodiments of this application;

[0038] Figure 4 This is a flowchart illustrating the flow control method provided in an embodiment of this application;

[0039] Figure 5 This is a schematic diagram of a flow control method provided in related technologies;

[0040] Figure 6 This is a schematic diagram of the architecture of the flow control system provided in an embodiment of this application;

[0041] Figure 7 This is a schematic diagram illustrating the application of flow control provided in the embodiments of this application;

[0042] Figure 8 This is a schematic diagram of the processing flow of the flow control device provided in the embodiments of this application;

[0043] Figure 9 This is a storage diagram illustrating the number of access requests provided in an embodiment of this application;

[0044] Figure 10This is a schematic diagram of the microservice processing flow provided in the embodiments of this application;

[0045] Figure 11 This is a schematic diagram of the architecture of the flow control system provided in the embodiments of this application. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0047] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0048] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0050] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0051] 1) In response to, used to indicate the conditions or states on which the operation performed depends. When the conditions or states on which it depends are met, one or more operations performed may be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.

[0052] 2) Microservices: A software development technique that is a variant of the service-oriented architecture style, which constructs an application as a set of loosely coupled services.

[0053] 3) Service-Oriented Architecture (SOA) is a component model that breaks down an application into different functional units (called services) and connects them through well-defined interfaces and protocols. These interfaces are defined in a neutral manner, independent of the hardware platform, operating system, and programming language used to implement the service. This allows services built on a wide variety of systems to interact in a unified and universal way.

[0054] 4) Load management: Management / planning / protection of the load of the microservice cluster, such as traffic control of the microservice cluster to prevent the entire microservice cluster from crashing due to external environment (such as cluster ingress traffic), internal service (or module) abnormalities or failures.

[0055] 5) Application Gateway (AGW), responsible for application layer routing and traffic control functions.

[0056] 6) Queries per second (QPS) is a measure of how much traffic a specific query server processes within a specified time. On the Internet, it is used to measure the performance of a machine that acts as a Domain Name System server.

[0057] 7) The Central Processing Unit (CPU) is the core of a computer's operation and control, and is the final execution unit for information processing and program execution.

[0058] 8) Memory is an important component of a computer, also known as internal memory or main memory. It is used to temporarily store data processed by the CPU and data exchanged with external storage devices such as hard drives.

[0059] The following describes the implementation scenarios of the flow control system provided in the embodiments of this application. See also Figure 1 , Figure 1 This is a schematic diagram of the architecture of a traffic control system 100 provided in an embodiment of this application. The traffic control system includes a traffic management service and at least one traffic control device corresponding to a business service cluster. To support an exemplary application, the traffic management terminal 400 is connected to the traffic management service 200 via a network 300. The traffic management service 200 is connected to the business service cluster traffic control device (traffic control device 600 is shown as an example) via the network 300. The traffic control devices are also connected to each other via a network. The network 300 can be a wide area network (WAN), a local area network (LAN), or a combination of both, and data transmission is achieved using wireless or wired links.

[0060] Traffic management terminal 400 is used to respond to quota configuration instructions for the business service cluster, determine the total traffic allocation quota for the business service cluster, and the total traffic allocation quota is used to indicate the number of access requests that the business service cluster can support. In actual implementation, the total traffic allocation quota can be manually entered and configured, or it can be predicted and automatically configured through machine learning models. The total traffic allocation quota of the business service cluster is synchronized to traffic management service 200.

[0061] Traffic management service 200 is used to receive the total traffic allocation quota of the business service cluster synchronized by traffic management terminal 400;

[0062] The flow control device 600 is used to send a request to obtain a target traffic allocation quota to the traffic management service when it receives a target access request for a business service cluster (such as one triggered by a user terminal). The target traffic allocation quota is used to indicate the maximum number of access requests that can be responded to within a unit of time.

[0063] Traffic management service 200 is also used to respond to acquisition requests, determine the target traffic allocation quota for the traffic control device based on the total traffic allocation quota and the number of historical access requests corresponding to each traffic control device, and send the target traffic allocation quota to the traffic control device 600.

[0064] The flow control device 600 is also used to determine that when the number of received target access requests does not reach the target flow allocation quota, the access operation corresponding to the target access request will be executed, that is, the target access request will be allowed to enter the business service cluster.

[0065] In some embodiments, the server in the traffic control system provided in this application (e.g., traffic management service 200, traffic control device 600, related business service cluster) can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0066] The traffic management terminal 400 can be a smartphone, tablet, laptop, desktop computer, smart voice interaction device (e.g., smart speaker), smart home appliance (e.g., smart TV), smartwatch, in-vehicle terminal, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, and this application embodiment does not impose any limitations on this. This application embodiment can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, and assisted driving.

[0067] In some embodiments, multiple servers can form a blockchain, with each server being a node on the blockchain. Information connections can exist between each node in the blockchain, allowing for information transmission between them. Data related to the traffic control system provided in this application embodiment (e.g., the total traffic allocation quota of the business service cluster, the number of historical access requests corresponding to the traffic control device, etc.) can be stored on the blockchain.

[0068] The following describes an electronic device for implementing a flow control method provided in an embodiment of this application. See also... Figure 2 , Figure 2 This is a schematic diagram of the structure of the electronic device 500 implementing the flow control method provided in the embodiments of this application. Taking the electronic device 500 as an example... Figure 1 Taking the flow control device shown as an example, the electronic device 500 implementing the flow control method provided in this application embodiment includes: at least one processor 510, a memory 550, at least one network interface 520, and a user interface 530. The various components in the electronic device 500 are coupled together through a bus system 540. It is understood that the bus system 540 is used to realize the connection and communication between these components. In addition to a data bus, the bus system 540 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 2 The general labeled all buses as Bus System 540.

[0069] The processor 510 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0070] Memory 550 may be removable, non-removable, or a combination thereof. Memory 550 may optionally include one or more storage devices physically located remote from processor 510. Memory 550 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 550 described in this application embodiment is intended to include any suitable type of memory.

[0071] In some embodiments, memory 550 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0072] Operating system 551 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0073] The network communication module 552 is used to reach other computing devices via one or more (wired or wireless) network interfaces 520, exemplary network interfaces 520 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0074] In some embodiments, the flow control device provided in this application can be implemented in software. Figure 2 A flow control device 553 stored in memory 550 is shown. It may be software in the form of programs and plug-ins, including the following software modules: a sending module 5531, a receiving module 5532, and a determining module 5533. These modules are logical and can therefore be arbitrarily combined or further divided according to the functions they implement. The functions of each module will be described below.

[0075] The traffic control system provided in the embodiments of this application is described below. The traffic control system provided in the embodiments of this application includes: a traffic management service, and at least one traffic control device corresponding to the service service cluster. See also... Figure 3 , Figure 3 This is a schematic diagram of the interaction process between devices in the flow control system provided in this application embodiment. The interaction process between devices in the flow control system provided in this application embodiment includes:

[0076] Step 101: When the traffic control device receives a target access request for the business service cluster, it sends a request to obtain the target traffic allocation quota to the traffic management service.

[0077] The target traffic allocation quota indicates the maximum number of access requests that can be responded to within a unit of time.

[0078] In practical applications, this service cluster is used to provide services for at least one business, such as gaming, media information recommendation, and online shopping. The service cluster can include multiple services, each implemented via physical machines or a cloud platform; it can be a microservice cluster. Users can send target access requests to the service cluster via their terminals to access the services provided. In this embodiment, the target access request first reaches the traffic control device, which forwards it to the service cluster, thus implementing traffic control for the service cluster during the forwarding process. At any given time, the traffic control device can receive one or more target access requests.

[0079] In practice, the traffic control device can be a gateway (such as an application gateway), a server, or other similar device. There can be one or more traffic control devices corresponding to a business service cluster (e.g., a traffic control device cluster). Each traffic control device can also control one or more business service clusters. A traffic management service can manage one or more traffic control devices.

[0080] Access requests sent by users through their terminals to the business service cluster must be forwarded by the traffic control device before reaching the cluster. This traffic control device controls the amount of access traffic (i.e., the number of access requests) entering the business service cluster based on a target traffic allocation quota. The target traffic allocation quota to be obtained in the request is the traffic allocation quota corresponding to the current time point of the target access request; that is, it indicates the maximum number of access requests allowed to be responded to within a unit of time corresponding to the current time point. In practical applications, this target traffic allocation quota can be pre-configured, predicted through a machine learning model, or calculated by the traffic management service based on preset logic, which will be explained later.

[0081] In some embodiments, the traffic control device is further configured to, when receiving a target access request for a business service cluster, obtain a request response quantity threshold corresponding to the traffic control device; and when the number of received target access requests does not reach the request response quantity threshold, send a request to obtain the target traffic allocation quota to the traffic management service.

[0082] In practical applications, when a flow control device sends a request to the flow management service to obtain a target traffic allocation quota, it can first obtain the request response count threshold corresponding to the flow control device itself. This threshold indicates the maximum number of access requests that the flow control device can support responding to. In actual implementation, this request response count threshold is determined based on the flow control device's own hardware and software computing resources (such as CPU, memory, network card, queue length, queue waiting time, etc.). Then, it obtains the number of received target access requests. Finally, it determines whether the number of received target access requests reaches the request response count threshold.

[0083] When the number of received target access requests does not reach the request-response threshold, a request to obtain the target traffic allocation quota is sent to the traffic management service. When the number of received target access requests reaches the request-response threshold, it indicates that the number of received target access requests has reached the maximum number that the traffic control device can handle. At this time, access to the target access requests is restricted, for example, by controlling the execution of access operations corresponding to some target access requests according to a restriction policy. This restriction policy can be based on factors such as business quota ratio, business priority, access user priority, and overall business performance.

[0084] In some embodiments, the flow control device is further configured to, when receiving a target access request for a service cluster, obtain the first sending time point corresponding to the last sending of the access request and the sending period of the access request; and when determining that a second sending time point has arrived based on the first sending time point and the sending period, send an access request for the target traffic allocation quota to the traffic management service.

[0085] In practical applications, when a traffic control device sends a request to the traffic management service to acquire a target traffic allocation quota, it can do so according to a specified sending cycle, such as 5 seconds, 10 seconds, etc., which can be set according to needs or experience. When the traffic control device receives a target access request for a business service cluster, it retrieves the first sending time point corresponding to the previous acquisition request and the sending cycle of the acquisition request. Based on the first sending time point and the sending cycle, it determines whether the second sending time point has arrived. If the second sending time point has arrived based on the first sending time point and the sending cycle, it sends the target traffic allocation quota acquisition request to the traffic management service. If the second sending time point has not arrived based on the first sending time point and the sending cycle, it can continue to use the target traffic allocation quota acquired in the previous acquisition request to control the access traffic (i.e., the number of access requests) entering the business service cluster. In this way, it can ensure that the traffic control device can control the business service cluster normally and avoid the traffic control device from malfunctioning or crashing due to excessive traffic.

[0086] Step 102: The traffic management service obtains the total traffic allocation quota for the business service cluster.

[0087] The total traffic allocation limit indicates the number of access requests that the business service cluster can support in responding to.

[0088] Here, the traffic management service can obtain the total traffic allocation limit for the business service cluster. In actual implementation, this total traffic allocation limit is the cluster's total traffic allocation limit, used to indicate the number of access requests (which can be the maximum number) that the business service cluster can support. This total traffic allocation limit can be manually configured or predicted and automatically configured based on a trained machine learning model. In practical applications, the traffic control system provided in this embodiment also includes a traffic management terminal (such as a terminal device). The traffic management terminal can be used to configure and calculate the total traffic allocation limit and synchronize it to the traffic management service, enabling the traffic management service to obtain the total traffic allocation limit for the business service cluster.

[0089] Step 103: In response to the request, the traffic management service determines the target traffic allocation quota for each traffic control device based on the total traffic allocation quota and the number of historical access requests corresponding to each traffic control device, and sends the target traffic allocation quota to the traffic control device.

[0090] In practical applications, after receiving a request, the traffic management service responds by determining the target traffic allocation quota for each traffic control device based on the total traffic allocation quota and the number of historical access requests corresponding to each traffic control device, and then sends the target traffic allocation quota to the traffic control device.

[0091] In some embodiments, the flow control device is further configured to report the number of received access requests to the flow management service according to the reporting period; the flow management service is further configured to, in response to the acquisition request, determine the target time period within the previous reporting period of the time point of the target access request; acquire the number of historical access requests reported by each flow control device within the target time period; and use the number of reports corresponding to each flow control device as the number of historical access requests corresponding to the corresponding flow control device.

[0092] In practical applications, traffic control devices report the number of received access requests to the traffic management service according to a reporting cycle. Based on this, the traffic management service can determine the target time period within the previous reporting cycle of the time point of the target access request when obtaining the number of historical access requests corresponding to each traffic control device. This target time period is the closest period within the previous reporting cycle to the time point of the target access request, such as the 30 seconds before the time point of the target access request. In this way, the completeness of the reported numbers can be guaranteed, and the number of the most recent historical access requests can be obtained, ensuring the accuracy of the calculation of the target traffic allocation quota.

[0093] In practical applications, the number of access requests reported by the flow control device is stored in the following way: obtaining historical access requests within the reporting period; determining the storage parameters within the reporting period based on the identifiers of the historical access requests within the reporting period; and storing the storage parameters in a memory structure, wherein the memory structure includes multiple memory units connected end to end.

[0094] For example, historical access requests include timestamps recorded by the flow control device within the reporting period. This can be understood as obtaining the timestamps of all historical access requests that have ever accessed the flow control device within the reporting period (e.g., 60 seconds). Based on the timestamps of the historical access requests to the flow control device within the reporting period (e.g., 60 seconds), storage parameters for the reporting period are determined. These storage parameters can represent the timestamps corresponding to all historical access requests to the flow control device within the reporting period, as well as the number of requests corresponding to each timestamp.

[0095] Continuing, the flow control device reads timestamps from multiple memory units within the reporting period. Based on the timestamps, it determines the number of requests within the reporting period and uses the sum of these requests as the number of historical access requests for the flow control device within the reporting period. For example, the memory structure stores storage parameters corresponding to historical access requests in multiple memory units. These storage parameters include the request time for the flow control device at a certain moment within the reporting period, and the number of requests corresponding to that time. The sum of the request counts corresponding to all moments within a certain period (such as the reporting period) is used as the total number of historical access requests reported by the flow control device within the reporting period, i.e., the reporting quantity.

[0096] In practical applications, the flow control device determines the storage parameters within the reporting period based on the identifiers of historical access requests within the reporting period in the following manner: the identifiers of historical access requests within the reporting period are used as the timestamps of the historical access requests within the reporting period; when there is a correlation between the identifier, the timestamp, and the memory unit index, the request count corresponding to the timestamp is incremented by 1; when there is no correlation between the identifier, the timestamp, and the memory unit index, the request count corresponding to the timestamp is reset to 1.

[0097] For example, the timestamps for historical access requests by the flow control device within the reporting period may include A1-A N (Time identifier is unique, therefore A1-A) N All of these are distinct values), and the time stamps (including A1-A) are... N The value of ) is used as the timestamp of historical access requests of the flow control device within the reporting period: timekey1-timekey N When the time identifier of an access request A within the reporting period for the flow control device is A1, the timestamp (timekey) is stored in the memory unit corresponding to the time identifier A1 and the current index X. x And after index X satisfies the equation under specific rules, the timestamp (timekey) of the memory unit corresponding to the current index X is stored. x The corresponding sum1 request count is incremented by 1. When the time identifier of one of the access requests A within the reporting period is A1, if the time identifier A1, the timestamp timekeyx stored in the memory unit corresponding to the current index X, and the index X cannot satisfy the equation under specific rule calculation, the timestamp timekeyx stored in the memory unit corresponding to the current index X is incremented. x After the corresponding sum1 request count is cleared to zero, it is reset to 1.

[0098] In practical applications, flow control devices store storage parameters in the memory structure in the following way: the remainder between the timestamp within the reporting period and the number of multiple memory units is used as the index of the memory unit; the timestamp and the number of requests corresponding to the timestamp are stored in the memory unit corresponding to the index.

[0099] For example, if there are N memory units, and the remainder when the timestamp of a historical access request for the flow control device within the reporting period is divided by N is 1, then the remainder of 1 when the timestamp is divided by the number of memory units is used as the index of the memory unit. Similarly, if the remainder when the timestamp of a historical access request for the flow control device within the reporting period is 2, then the remainder of 2 when the timestamp is divided by the number of memory units is used as the index of the memory unit. Since the timestamp is determined by the timestamp of the historical access request, at a target time within the reporting period, there may be multiple historical access requests that accessed the flow control device at that target time. The request count corresponding to the timestamp represents the number of times these multiple historical access requests accessed the flow control device at that target time.

[0100] In some embodiments, the traffic management service is further configured to, in response to an acquisition request, determine the total number of historical access requests corresponding to each traffic control device; determine the ratio of the number of historical access requests corresponding to the traffic control device to the total number; and use the product of the total traffic allocation quota of the service cluster and the ratio as the target traffic allocation quota of the traffic control device.

[0101] In practical applications, for each traffic control device, the traffic management service responds to the request by performing the following processing: First, it calculates the total number of historical access requests corresponding to each traffic control device, then determines the ratio of the number of historical access requests corresponding to that traffic control device to the total number, and then uses the product of the total traffic allocation quota of the business service cluster and the ratio as the target traffic allocation quota of the traffic control device.

[0102] In some embodiments, the traffic management service is further configured to, in response to an acquisition request, acquire the number of traffic control devices when the number of historical access requests corresponding to each traffic control device is zero; average the total traffic allocation amount according to the number of devices to obtain a processing result; and use the processing result as the target traffic allocation amount for the traffic control devices.

[0103] In practical applications, when the number of historical access requests corresponding to each traffic control device is zero, that is, when the traffic control device is reset or used for the first time, the traffic management service can also respond to the request to obtain the number of traffic control devices; according to the number of devices, the total traffic allocation quota is averaged, and the result of the averaged processing is used as the target traffic allocation quota for the traffic control device.

[0104] By applying the above embodiments, the traffic management service can reasonably allocate the target traffic allocation quota entering the business service cluster based on the total traffic allocation quota of the business service cluster, thereby improving the rationality and efficiency of traffic control.

[0105] Step 104: When the number of target access requests received by the flow control device does not reach the target flow allocation quota, it determines that the access operation corresponding to the target access request will be executed.

[0106] In practical applications, the flow control device receives the target flow allocation quota returned by the flow management service, then obtains the number of received target access requests, and determines whether the number of received target access requests has reached the target flow allocation quota. When it is determined that the number of received target access requests has not reached the target flow allocation quota, the device will execute the access operation corresponding to the target access request, that is, allow the target access request to access the business service cluster. When it is determined that the number of received target access requests has reached the target flow allocation quota, the device will restrict the execution of the access operation corresponding to the target access request, that is, disallow some target access requests from entering the business service cluster, i.e., do not execute the access operation corresponding to some target access requests. This ensures that the number of target access requests entering the business service cluster is lower than the number of access requests that the business service cluster can support, thus achieving flow control at the entry point of the business service cluster.

[0107] In some embodiments, the flow control device is further configured to: determine at least one target service corresponding to the target access request when the number of received target access requests does not reach the target flow allocation quota; send a first request to obtain the service flow allocation quota to the flow management service for each target service, wherein the service flow allocation quota indicates the maximum number of access requests allowed to respond to the target service within a unit of time; correspondingly, the flow management service is further configured to: in response to the first request, determine the service flow allocation quota of the flow control device for each target service based on the total service flow allocation quota of the target service and the number of historical access requests for the target service corresponding to each flow control device; send the service flow allocation quota to the flow control device, wherein the total service flow allocation quota indicates the number of access requests required by the target service; correspondingly, the flow control device is further configured to: determine that the access operation corresponding to the first target access request will be executed when the number of the first target access requests corresponding to the target service in the target access requests does not reach the service flow allocation quota.

[0108] In practical applications, after performing traffic verification on the target traffic allocation quota (i.e., determining whether the number of received target access requests has reached the target traffic allocation quota), traffic verification can also be performed on the business traffic allocation quota of each target service corresponding to the target access request. That is, determining whether the number of target access requests for each corresponding target service has reached the business traffic allocation quota, so as to perform corresponding business traffic control based on the business traffic allocation quota.

[0109] In some embodiments, the flow control device is further configured to: determine at least one target object corresponding to the first target access request when the number of first target access requests corresponding to the target service does not reach the service traffic allocation quota; send a second acquisition request to the traffic management service for each target object, the second acquisition request being used to acquire the number of historical access requests that have been responded to for the target object; correspondingly, the traffic management service is further configured to acquire the object traffic allocation quota of the target object, the object traffic allocation quota being used to indicate the maximum number of access requests that are allowed to be responded to for the target object; and, in response to the second acquisition request, acquire the number of historical access requests that have been responded to for each target object, and send the number of responded requests to the flow control device; correspondingly, the flow control device is further configured to: determine that the access operation corresponding to the second target access request will be executed for each target object when the number of second target access requests corresponding to the target object in the first target access request does not reach the number of responded requests.

[0110] In practical applications, after performing traffic verification on the business traffic allocation quota for each target service corresponding to the target access request, there is also a traffic verification on the object traffic allocation quota for each target object corresponding to the access request. That is, it is determined whether the number of historical access requests responded to by the target object has reached the object traffic allocation quota, so as to perform corresponding object traffic control according to the object traffic allocation quota.

[0111] In practical applications, the aforementioned business traffic allocation quotas and object traffic control can be configured manually or automatically based on predictions from trained machine learning models.

[0112] In some embodiments, the flow control device is further configured to obtain the access quality score corresponding to the service cluster when the number of received target access requests does not reach the target traffic allocation quota; and to determine that the access operation corresponding to the target access request will be executed when the access quality score is greater than the first quality score threshold.

[0113] In practical applications, the flow control device can further obtain the access quality score corresponding to the service cluster when the number of received target access requests does not reach the target traffic allocation quota. This access quality score is used to indicate the access response quality of the service cluster within the target time period. When the obtained access quality score is greater than a first quality score threshold, it is determined that the access operation corresponding to the target access request will be executed.

[0114] In some embodiments, the flow control device is further configured to obtain access quality assessment information of the service cluster, the access quality assessment information including at least one of the following: access response time and response error probability; and determine the access quality score corresponding to the service cluster based on the access quality assessment information.

[0115] In practical applications, flow control devices can obtain the access quality score corresponding to the service cluster in the following way: First, obtain the access quality assessment information of the service cluster, which includes at least one of the following: access response time and response error probability. Specifically, access response time and access quality score are inversely proportional, and response error probability is also inversely proportional to access quality score.

[0116] Therefore, based on the access quality assessment information, the access quality score corresponding to the business service cluster is determined. When the access quality score is determined based on both access response time and response error probability, corresponding weight values ​​can be set for access response time and response error probability. Thus, a weighted calculation is performed based on access response time, response error probability, and the corresponding weight values ​​to determine the access quality score.

[0117] In some embodiments, the flow control device is further configured to determine that when the access quality score is less than a first quality score threshold and the access quality score is greater than a second quality score threshold, to execute an access operation corresponding to a portion of the target access request; and to determine that when the access quality score is less than the second quality score threshold, to restrict the execution of the access operation corresponding to the target access request.

[0118] In practical applications, in addition to setting a first quality score threshold, a second quality score threshold lower than the first quality score threshold can also be set. When the access quality score is less than the first quality score threshold and greater than the second quality score threshold, the access operations corresponding to a portion of the target access requests can be executed according to a preset execution probability. This ensures partial response to access requests, improves the utilization of computing resources, and ensures timely response to access requests, thus guaranteeing user experience. When the access quality score is less than the second quality score threshold, the execution of the access operations corresponding to the target access requests will be restricted. In practical applications, the execution of the access operations corresponding to the target access requests can be completely restricted to provide recovery time for the business service cluster and assist the business service cluster in recovering quickly.

[0119] In practical applications, after restricting the access operations corresponding to the target access request, the flow control device can perform access probes on the business service cluster according to a preset time period, so as to determine the recovery of the business service cluster in a timely manner and control the target access request to perform the corresponding access operations.

[0120] In some embodiments, the business service cluster includes multiple business services, at least two of which form a service call chain to implement the corresponding business. The system also includes: a cluster traffic control service corresponding to each business service; the cluster traffic control service is used to obtain a request response quantity threshold corresponding to the first business service when the traffic control device controls a target access request to enter the business service cluster; when the number of target access requests passing through the first business service does not reach the request response quantity threshold, it determines that the access operation of the first business service corresponding to the target access request will be executed; when the number of target access requests passing through the first business service reaches the request quantity threshold, it determines that the execution of the access operation of the first business service corresponding to the target access request will be restricted.

[0121] In practical applications, after the traffic control device controls a target access request to enter the business service cluster, the business service cluster can also perform traffic control on the incoming target access request. Based on this, in this embodiment, the traffic control system further includes a cluster traffic control service. This cluster traffic control service can be an API deployed on the physical machine or cloud platform providing the business service, or it can be a proxy server (i.e., an agent) independent of the physical machine or cloud platform providing the business service. There can be a one-to-one correspondence between this cluster traffic control service and the business services in the business service cluster.

[0122] In practical implementation, the cluster traffic control service is used to obtain the request-response quantity threshold of the first business service corresponding to the cluster traffic control service when the traffic control device controls the target access request to enter the business service cluster. This request-response quantity threshold is determined based on the traffic control device's own hardware and software computing resources (such as CPU, memory, network card, queue length, queue waiting time, etc.). Then, the number of target access requests passing through the first business service is obtained, and it is determined whether the number of target access requests passing through the first business service reaches the request-response quantity threshold.

[0123] When the number of target access requests passing through the first business service does not reach the request-response threshold, the access operation corresponding to the first business service for the target access request will be executed; when the number of target access requests passing through the first business service reaches the request threshold, the execution of the access operation corresponding to the first business service for the target access request will be restricted. This allows for traffic control of each business service within the business cluster, further ensuring the security of business services within the business service cluster and improving the stability of the business service cluster.

[0124] In some embodiments, the cluster traffic control service is further configured to, after restricting the access operation of the first service corresponding to the target access request, report a traffic overload notification message of the first service to the traffic control device through the service call chain; the traffic control device is further configured to, upon receiving the traffic overload notification message, determine that the service access request corresponding to the first service will be restricted.

[0125] In practical applications, after restricting access operations to the primary service corresponding to the target access request, the cluster traffic control service can also report a traffic overload notification message for the primary service to the traffic control device through the service call chain, thereby notifying the traffic control device that the primary service is experiencing traffic overload. Correspondingly, upon receiving the traffic overload notification message, the traffic control device determines to restrict the service access requests corresponding to the primary service. In actual implementation, this can be done by completely restricting the service access requests corresponding to the primary service until the primary service recovers.

[0126] In practical applications, after the flow control device restricts the business access requests corresponding to the first business service, it can perform access probes on the first business service according to a preset time period to determine in a timely manner whether the first business service has been restored and to control the access of the corresponding business access requests.

[0127] In some embodiments, the cluster traffic control service corresponding to the second service is further configured to determine the third service that is next to the second service in the service call chain, wherein the second service is a non-last service in the service call chain; obtain the service quality score of the third service; when the service quality score is lower than the quality score threshold, report a service exception notification message of the third service to the traffic control device through the service call chain; the traffic control device is further configured to receive the service exception notification message; and determine to restrict the service access requests corresponding to the third service.

[0128] In practical applications, the cluster traffic control service corresponding to a non-last business service in the service call chain can also evaluate the service quality of downstream business services. For example, the second business service, which is not the last business service, can identify the third business service in the service call chain that is next to the second business service, and then obtain the service quality score of the third business service. The process of determining the service quality score can be the same as the process of determining the access quality score described above, and will not be repeated here.

[0129] When the service quality score falls below the quality score threshold, a service anomaly notification message for the third service is reported to the traffic control device through the service call chain, notifying the traffic control device that the third service has experienced a service anomaly. Upon receiving the service anomaly notification message, the traffic control device determines to restrict access requests to the service corresponding to the third service. In practice, access requests to the service corresponding to the third service can be completely restricted until the third service recovers.

[0130] In practical applications, after the flow control device restricts the business access requests corresponding to the third business service, it can perform access probes on the third business service according to a preset time period to determine the recovery of the third business service in a timely manner and control the access of the corresponding business access requests.

[0131] Applying the above embodiments of this application, the traffic management service reasonably allocates a target traffic allocation quota (indicating the maximum number of access requests allowed to be responded to within a unit of time) to the business service cluster based on the total traffic allocation quota of the business service cluster (used to indicate the number of access requests that the business service cluster can support). Thus, the traffic control device that controls access requests entering the business service cluster performs traffic control on the access requests entering the business service cluster according to the allocated target traffic allocation quota. That is, when the traffic control device determines that the number of received access requests has not reached the target traffic allocation quota, it determines that the access operation corresponding to the target access request will be executed, that is, the target access request is allowed to access the business service cluster.

[0132] In this way, by controlling the ingress traffic of the business service cluster, overall traffic control of the business service cluster can be achieved, improving the utilization rate of service computing resources and enhancing the rationality and efficiency of traffic control. At the same time, it reduces the occurrence of service anomalies due to excessive instantaneous load, thus improving the effectiveness of traffic control. Based on this, the stability of the business service cluster is improved.

[0133] The following describes the traffic control method provided in the embodiments of this application. The traffic control method provided in the embodiments of this application is applied to a traffic control system including a traffic management service and at least one traffic control device corresponding to a service service cluster. See also... Figure 4 , Figure 4This is a flowchart illustrating the flow control method provided in this application embodiment. The flow control method provided in this application embodiment includes:

[0134] Step 201: When the traffic control device receives a target access request for the business service cluster, it sends a request to obtain the target traffic allocation quota to the traffic management service.

[0135] The request is used by the traffic management service to respond to the request, determine the target traffic allocation quota of the traffic control device based on the total traffic allocation quota of the business service cluster and the number of historical access requests corresponding to each traffic control device, and return the result.

[0136] The total traffic allocation quota indicates the number of access requests that the business service cluster can support, while the target traffic allocation quota indicates the maximum number of access requests allowed to be responded to within a unit of time.

[0137] In practical applications, this service cluster is used to provide services for at least one business, such as gaming, media information recommendation, and online shopping. The service cluster can include multiple services, each implemented via physical machines or a cloud platform; it can be a microservice cluster. Users can send target access requests to the service cluster via their terminals to access the services provided. In this embodiment, the target access request first reaches the traffic control device, which forwards it to the service cluster, thus implementing traffic control for the service cluster during the forwarding process. At any given time, the traffic control device can receive one or more target access requests.

[0138] In practice, the traffic control device can be a gateway (such as an application gateway), a server, or other similar device. There can be one or more traffic control devices corresponding to a business service cluster (e.g., a traffic control device cluster). Each traffic control device can also control one or more business service clusters. A traffic management service can manage one or more traffic control devices.

[0139] Access requests sent by users through their terminals to the business service cluster must be forwarded by the traffic control device before reaching the cluster. This traffic control device controls the amount of access traffic (i.e., the number of access requests) entering the business service cluster based on a target traffic allocation quota. The target traffic allocation quota to be obtained in the request is the traffic allocation quota corresponding to the current time point of the target access request; that is, it indicates the maximum number of access requests allowed to be responded to within a unit of time corresponding to the current time point. In practical applications, this target traffic allocation quota can be pre-configured, predicted through a machine learning model, or calculated by the traffic management service based on preset logic, which will be explained later.

[0140] The traffic management service can obtain the total traffic allocation quota of the business service cluster. In actual implementation, this total traffic allocation quota is the cluster's total traffic allocation quota, used to indicate the number of access requests (which can be the maximum number) that the business service cluster can support. This total traffic allocation quota can be manually configured or predicted and automatically configured based on a trained machine learning model. In practical applications, the traffic control system provided in this embodiment also includes a traffic management terminal (such as a terminal device). The traffic management terminal can be used to configure and calculate the total traffic allocation quota and synchronize it to the traffic management service, enabling the traffic management service to obtain the total traffic allocation quota of the business service cluster. In practical applications, after receiving an acquisition request, the traffic management service responds to the acquisition request by determining the target traffic allocation quota for each traffic control device based on the total traffic allocation quota and the number of historical access requests corresponding to each traffic control device, and sends the target traffic allocation quota to the traffic control device.

[0141] Step 202: Receive the target traffic allocation quota based on the retrieval request.

[0142] Step 203: When the number of received target access requests does not reach the target traffic allocation quota, determine that the access operation corresponding to the target access request will be executed.

[0143] In practical applications, the flow control device receives the target flow allocation quota returned by the flow management service, then obtains the number of received target access requests, and determines whether the number of received target access requests has reached the target flow allocation quota. When it is determined that the number of received target access requests has not reached the target flow allocation quota, the device will execute the access operation corresponding to the target access request, that is, allow the target access request to access the business service cluster. When it is determined that the number of received target access requests has reached the target flow allocation quota, the device will restrict the execution of the access operation corresponding to the target access request, that is, disallow some target access requests from entering the business service cluster, i.e., do not execute the access operation corresponding to some target access requests. This ensures that the number of target access requests entering the business service cluster is lower than the number of access requests that the business service cluster can support, thus achieving flow control at the entry point of the business service cluster.

[0144] Applying the above embodiments of this application, the traffic management service reasonably allocates a target traffic allocation quota (indicating the maximum number of access requests allowed to be responded to within a unit of time) to the business service cluster based on the total traffic allocation quota of the business service cluster (used to indicate the number of access requests that the business service cluster can support). Thus, the traffic control device that controls access requests entering the business service cluster performs traffic control on the access requests entering the business service cluster according to the allocated target traffic allocation quota. That is, when the traffic control device determines that the number of received access requests has not reached the target traffic allocation quota, it determines that the access operation corresponding to the target access request will be executed, that is, the target access request is allowed to access the business service cluster.

[0145] In this way, by controlling the ingress traffic of the business service cluster, overall traffic control of the business service cluster can be achieved, improving the utilization rate of service computing resources and enhancing the rationality and efficiency of traffic control. At the same time, it reduces the occurrence of service anomalies due to excessive instantaneous load, thus improving the effectiveness of traffic control. Based on this, the stability of the business service cluster is improved.

[0146] The following example, using a business service cluster implemented through a microservice cluster, illustrates an exemplary application of this application's embodiments in a real-world application scenario.

[0147] With the development of the internet, business service clusters (such as game services) have gradually evolved into microservice architectures. Microservice architectures can lower the development threshold and increase the speed of iterative development. However, microservice architectures introduce some new problems: the architecture is complex, it is not easy to scale quickly, and if a critical system path fails, it may cause the entire system to crash. The technology selection of services within a microservice architecture is complex, with varying performance and robustness. Downstream services with poor performance and robustness may drag down high-performance upstream services, leading to the collapse of the entire microservice architecture.

[0148] See Figure 5 , Figure 5 This is a schematic diagram of a traffic control method provided in related technologies. Here, related technologies typically employ the following scheme for traffic control of microservice clusters: When external access traffic (i.e., access requests to the microservice cluster) enters the microservice cluster, each service (machine) within the cluster performs load protection based on its own hardware and software resource consumption (including CPU / memory / network card). If the current consumption of computing resources (i.e., computing resources responding to access requests) reaches a preset threshold, then the service of that access traffic on the current machine is restricted. However, the traffic control scheme in related technologies has the following problems:

[0149] (1) Lack of intuitiveness and difficulty in setting thresholds. For example, thresholds such as 70% CPU and 80% memory are difficult to map to specific business implications. These thresholds cannot indicate how many QPS or other business-interesting parameters the machine can provide, making threshold setting difficult. (2) Relatively passive, lacking overall system coordination, and prone to wasting computing resources. For example... Figure 5 As shown, a complete business logic needs to go through four services in sequence: A->C->B->D. When B is overloaded but C is not overloaded (there may still be a lot of spare resources), each time traffic reaches B, it is restricted. The traffic cannot complete the normal business logic, resulting in the waste of C's computing resources. (3) It is not conducive to business resource sharing. In the production environment, multiple businesses often share computing resources. Restricting traffic only according to computing resources may cause the computing resources to be used in an unreasonable way. For example, if two businesses, a and b, share computing resources with a resource consumption ratio of 9:1, the traffic of business a changes greatly, which can easily cause the entire computing resources to be overloaded. If only the computing resources are restricted, business b, which consumes less resources and has a smoother traffic, may be restricted. (4) It is difficult and costly to transform the microservice cluster as a whole. The technology selection and historical baggage of each service in the microservice cluster are different. Some old services on the critical path may not be easy to change, not easy to add service protection themselves, and not easy to replace in a short time. Therefore, the microservice cluster needs to provide protection for the service, which cannot be achieved in the relevant technologies.

[0150] Based on this, this application provides a traffic control system, which is an upstream and downstream load management solution that can ensure the overall security of microservices. This traffic control system can be applied to microservice clusters deployed on physical machines, as well as microservice clusters deployed on cloud platforms.

[0151] See Figure 6 , Figure 6This is a schematic diagram of the architecture of the flow control system provided in this application embodiment. The flow control system provided in this application embodiment includes: a flow control management terminal, a flow management service, an AGW (i.e., the aforementioned flow control device), and an elastic circuit breaker API (i.e., the aforementioned cluster flow control service). There can be multiple AGWs, i.e., an AGW cluster.

[0152] Here, (1) the traffic control management terminal is responsible for calculating the total traffic allocation quota for clusters / businesses / users, and then distributes the calculated total quota to the traffic management service. (2) AGW and the traffic management service perform traffic control at the entry point of the microservice cluster. (3) The elastic circuit breaker API protects downstream services. (4) When a downstream service is abnormal, it can be passed up the service call chain level by level.

[0153] In practical applications, the traffic control management terminal is used by product personnel, primarily responsible for configuring the total cluster traffic allocation quota (SQ), the total business traffic allocation quota (BQ), and the total object traffic allocation quota (UQ), and synchronizing them to the traffic management service. Taking the allocation of the total cluster traffic allocation quota (i.e., the aforementioned total traffic allocation quota) as an example, the traffic management service calculates the current traffic allocation quota (i.e., the aforementioned target traffic allocation quota) of each AGW in the microservice cluster in real time based on the total cluster traffic allocation quota and the historical access data reported by the AGW (i.e., the aforementioned number of historical access requests). When the AGW reports historical access data, the calculated current traffic allocation quota is returned as a response to the corresponding AGW. The AGW controls the ingress traffic (i.e., access requests to the microservice cluster) of the microservice cluster based on the received current traffic allocation quota.

[0154] The embodiments of this application can be applied to game activities on game clients, effectively ensuring the secure operation of game activities. See also... Figure 7 , Figure 7 This is a schematic diagram illustrating the application of flow control provided in an embodiment of this application. Here, as... Figure 7 As shown in (1), this indicates that the game client had two hourly game activities at 11:00:00 and 19:00:00, with the traffic exceeding the preset quota at 11:00, triggering traffic control. Figure 7 As shown in Figure (2), this is a detailed diagram of traffic restriction when traffic control is triggered: line a represents the total ingress traffic, line b represents the restricted traffic, and line c represents the traffic that enters the microservice cluster normally without being restricted. In actual implementation, the traffic of such game activities is generally unpredictable in advance, making it impossible to accurately deploy sufficient computing resources. Moreover, the activity time is short, and dynamic expansion is not timely. By applying the traffic control system provided in this application, AGW can successfully restrict traffic exceeding business needs outside the microservice cluster, avoiding service anomalies with varying performance and robustness in the downstream microservice cluster, thereby ensuring the safe operation of the activity.

[0155] See also Figure 6 The present application provides a detailed description of the flow control system provided in its embodiments, including:

[0156] First, the traffic control management terminal is used by product personnel, who are primarily responsible for configuring the total cluster traffic allocation quota (SQ), the total business traffic allocation quota (BQ), and the total object traffic allocation quota (UQ), and synchronizing these quotas with the traffic management service.

[0157] (1) Total traffic allocation limit (SQ): The maximum traffic allocation limit that the microservice cluster where the business is located can handle, i.e., the maximum QPS (since multiple businesses often share the computing resources of the microservice cluster in the production environment, it is necessary to set the maximum allocation limit of the microservice cluster).

[0158] (2) Total Business Traffic Allocation Quota (BQ): The maximum allocation quota for the business traffic required. Traffic exceeding this maximum allocation quota can be restricted for the business.

[0159] (3) Total traffic allocation limit (UQ): The maximum number of times a user can access a certain service within a unit of time (such as the access duration of the service). User traffic exceeding this maximum number of accesses needs to be restricted.

[0160] Second, external access traffic must pass through AGW before it can access the microservice cluster. See also Figure 8 , Figure 8 This is a schematic diagram of the processing flow of the flow control device AGW provided in the embodiments of this application, including:

[0161] Step 301: The AGW first checks whether the received access traffic exceeds the traffic threshold of its own hardware and software (CPU / Memory / Network Card / Queue Length / Queue Waiting Time, etc.). If it exceeds, the access traffic is limited; otherwise, proceed to step 301. In practical applications, the traffic thresholds for these AGW computing resources are independent of the business logic and can be set uniformly.

[0162] Step 302: AGW checks the cluster traffic allocation quota: Based on the target traffic allocation quota (i.e., the cluster traffic allocation quota) calculated by the traffic management service, it determines whether the received access traffic exceeds the target traffic allocation quota. If it exceeds, the access traffic is restricted; otherwise, proceed to step 303. The process by which AGW obtains the target traffic allocation quota includes:

[0163] (a) When external traffic enters the AGW, the AGW counts all traffic using the machine identifier (usually IP) as the key and checks the request volume of historical access requests within the most recent target time period in memory. If the request volume is greater than the local machine's target traffic allocation quota, the traffic is restricted. If the request volume is less than the target traffic allocation quota or the target traffic allocation quota is zero (the initial quota is zero), the access traffic is unrestricted.

[0164] (b) The specific process by which AGW saves the number of access requests for the microservice cluster into memory is as follows: Figure 9 This is a storage diagram illustrating the number of access requests provided in an embodiment of this application, wherein, Figure 9 Figure (1) is a schematic diagram of the storage principle. Figure 9 Figure (2) is a schematic diagram of the storage strategy. Here, combined with... Figure 9 As shown in (1) and (2), the memory cell (i.e., memory unit) where the access request falls is calculated based on the request time of the access request. Then, the count of that memory cell is incremented by one. These six cells are connected end to end to form a loop. As the time window moves, the cell at index 5 is used up, and then the cell at index 0 is used.

[0165] The specific implementation is as follows: First, AGW allocates an array Arr[6] of length 6. Each item in this array is a structure, which mainly contains two members:

[0166] time_t timekey; / / time key

[0167] int64_t sum; / / Request a count

[0168] Then, for each access request, obtain the number of seconds A (A represents the complete number of seconds). Use A / 10 (between [0, 5]) as the array index and calculate timekey = AA%60 + index*10. If timekey equals Arr[index].timekey, then increment Arr[index].sum by 1. If timekey does not equal Arr[index].timekey, it means that a loop has occurred (a loop means that a set of Arr[index] has been filled). In this case, directly set Arr[index].sum equal to 1 (i.e., clear it and then increment sum by 1), and reset Arr[index].timekey to the calculated timekey.

[0169] (c) Each time traffic enters, AGW checks whether the time interval since the last request for the target traffic allocation quota has reached 5 seconds. If it has reached 5 seconds, it reports the number of its own access requests (i.e., the array Arr above) and the local machine (i.e., AGW) identifier to the traffic management service, and the traffic management service returns the target traffic allocation quota.

[0170] (d) The traffic management service receives the reported traffic volume from each AGW and stores the reported traffic volume for the past few time slices. It then takes the data statistics from the first 30 seconds (ensuring the completeness of the AGW reported data within this calculation period) and calculates the target traffic allocation quota for each AGW according to the normal traffic allocation ratio. The calculation method for the target traffic allocation quota for each AGW is as follows:

[0171] Calculate the total number of access requests reported by all AGWs in the first 30 seconds (total); calculate the total number of access requests reported by this AGW in the first 30 seconds (self); the current target traffic allocation quota for this AGW is: quota(total cluster traffic allocation quota) * self / total.

[0172] (e) The traffic management service returns the target traffic allocation quota for each AGW, and also returns the number of all reporting AGWs. In some abnormal situations, the AGW uses the average quota (total traffic allocation quota of the cluster / number of all reporting AGWs) to limit traffic.

[0173] Step 303: AGW checks the service traffic allocation quota: Based on the service traffic allocation quota calculated by the traffic management service, it determines whether the received access traffic exceeds the service traffic allocation quota. If it exceeds, the access traffic is restricted; otherwise, proceed to step 304. The process of AGW obtaining the service traffic allocation quota is similar to the process of AGW obtaining the target traffic allocation quota. The main differences are as follows, as shown in Table 1.

[0174] Table 1

[0175]

[0176] In practical applications, AGWs of multiple microservice clusters can share the traffic management service, provided that the following conditions are met: 1. Businesses are not deployed across AGW clusters; 2. The traffic management service can determine the microservice cluster corresponding to each AGW based on the configuration; 3. When calculating the target traffic allocation quota and business traffic allocation quota, the traffic management service calculates the AGWs of each microservice cluster independently.

[0177] Step 304: the AGW checks the total quota of target traffic allocation: determining whether the number of responded requests exceeds the total quota of target traffic allocation according to the number of responded user historical access requests calculated by the traffic management service. If the number is exceeded, the current access traffic is restricted; if the number is not exceeded, step 305 is performed.

[0178] Wherein, the process for the AGW to obtain the number of responded requests includes: when external access traffic enters the AGW, the AGW accesses the traffic management service, that is, sends the "service + user ID" corresponding to the access traffic to the traffic management service; the traffic management service takes "service + user ID" as the key, counts the number of responded historical access requests of the user for the service, and returns the statistical result to the AGW.

[0179] Step 305: the AGW evaluates the access quality of downstream services, and according to the downstream services (such as Figure 6 Service A shown) in a target time period (such as the last 30 seconds), determines whether it is necessary to restrict the current access traffic.

[0180] In practical applications, the access quality can be based on the average downstream time consumption (i.e., Figure 8 response time shown) / downstream response error distribution (i.e., Figure 8 error distribution shown) and other parameters for evaluation. The evaluation objects of downstream access quality can be machines / interfaces / services, etc. Therefore, whether it is necessary to restrict the current access traffic can be determined according to the preset access quality thresholds (such as average time consumption threshold, probability threshold of error distribution, etc.).

[0181] Taking the average downstream time consumption (AT) as an example, two average time consumption thresholds HA and HB are set, where HA>HB, and traffic control can be implemented according to the following strategy: 1. When AT<HB, allow normal access to the downstream (that is, do not restrict the current access traffic); when HB<AT<HA, access the downstream according to a certain probability, and the larger the AT is, the smaller the access probability is; when AT>HA, completely restrict access to the downstream (that is, restrict the current access traffic), and after an interval of a certain period of time, when external access traffic enters again, re-probe and access the downstream to determine whether the access quality is restored; if restored, continue the access; if not restored, continue to restrict the access, and a service abnormality message can also be reported to notify the staff for maintenance.

[0182] Thirdly, after the access traffic enters the microservice cluster, the traffic control logic of each microservice in the microservice cluster is as shown in Figure 10 shown, Figure 10 is a schematic diagram of the processing flow of the microservice provided by the embodiment of the present application, which includes:

[0183] Step 401: Each microservice first checks whether the received access traffic exceeds the traffic threshold supported by its own hardware and software (CPU / Memory / Network Card / Queue Length / Queue Waiting Time, etc.). If it exceeds, the access traffic is limited; otherwise, proceed to step 402.

[0184] Step 402: Each microservice evaluates the access quality of downstream services, such as evaluating the access quality of downstream services within a target time period (e.g., within the last 30 seconds), and determines whether the downstream services are abnormal.

[0185] In practical applications, access quality can be determined based on the average latency of downstream devices (e.g., ...). Figure 10 (Response duration shown) / Downstream response error distribution (e.g.) Figure 10 The downstream access quality is evaluated using parameters such as the error distribution shown. The evaluation object can be a machine / interface / service, etc. Therefore, it is possible to determine whether the downstream service is abnormal based on pre-set access quality thresholds (such as average latency thresholds, error distribution probability thresholds, etc.).

[0186] Step 403: When a microservice discovers that its own computing resources are overloaded (i.e., the traffic is too high) or that a downstream service is abnormal, it passes the corresponding notification message layer by layer through the service call chain until it reaches the AGW.

[0187] See also Figure 6 When D detects that its computing resources are overloaded, it sends a notification message from B to D, and then this message travels along the reverse call chain (dotted line in the diagram: D->B->A->AGW) all the way back to AGW. When AGW detects a service anomaly within the microservice cluster, it restricts access traffic to the corresponding service. After a certain interval, when external access traffic re-enters, it re-probes downstream access to determine if the access quality has recovered. If it has, access continues; if not, access remains restricted, and a service anomaly message can be reported to notify maintenance personnel.

[0188] In practice, the traffic control logic within the microservice cluster can be implemented by setting up an elastic circuit breaker API on the machine or container where the microservices are deployed. In other embodiments, an agent service can be installed on the machine or container where the microservices are deployed, allowing the traffic control logic within the microservice cluster to be implemented through the agent. See also Figure 11 , Figure 11This is a schematic diagram of the traffic control system architecture provided in this application embodiment. Here, when traffic control is performed within a microservice cluster, access traffic first passes through the Agent before reaching each microservice in the microservice cluster. Thus, 1. the existing microservice cluster does not need to be modified to achieve internal traffic control for the entire microservice cluster; 2. upgrades to traffic control-related features are largely unrelated to the microservice cluster, facilitating changes.

[0189] In practical applications, (1) the above data dimensions are not limited to clusters, businesses and users, but can also include data dimensions such as time, such as the number of access requests per week; (2) the allocation quotas for data dimensions such as clusters, businesses and users can be predicted and automatically configured based on the machine learning model trained, thereby improving configuration efficiency and reducing the waste of human resources; (3) when the computing resources of the microservice cluster are overloaded, traffic can be limited according to certain strategies, such as according to business quota ratio, business priority, access user priority, and overall business performance.

[0190] Applying the above embodiments of this application, (1) the threshold setting is simple and business-meaningful: only the total allocation of data dimensions such as cluster / business / user needs to be set, usually the QPS and other indicators that are easy for the business to understand. The thresholds for computing resources and downstream service quality are universal, unrelated to the business, and do not need to be changed frequently. (2) Traffic control is performed on the entire microservice cluster, rather than single-point control. For example, in the service call chain of business W: A->B->C->D, if a node fails, AGW will restrict the entire service call chain of business W, restricting overloaded traffic from entering the microservice cluster, avoiding waste of computing resources in the microservice cluster, and gaining time for the microservice cluster to recover quickly. (3) It is convenient to be compatible with existing microservice clusters. For example, in the service call chain of business W: A->B->C->D, D is a key node, and D is inconvenient to change to access the elastic circuit breaker API for traffic control for various reasons. Then, the upstream B / C of D can be used to evaluate the access quality of D, and then it can be determined whether the traffic entering D needs to be restricted based on the access quality. (4) Because it can limit the traffic of the entire microservice cluster, it is convenient for different businesses to share computing resources.

[0191] It is understood that in the embodiments of this application, data such as user information are involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0192] The following describes the implementation of the flow control device 553 provided in this application embodiment as an exemplary structure of a software module. The flow control device 553 provided in this application embodiment can be disposed in a flow control device, which is located within a flow control system. The flow control system includes a flow management service and at least one flow control device corresponding to a service service cluster.

[0193] In some embodiments, such as Figure 2 As shown, the software modules stored in the flow control device 553 in the memory 550 may include: a sending module 5531, configured to send a target traffic allocation quota acquisition request to the traffic management service when a target access request for the service cluster is received; wherein, the acquisition request is configured for the traffic management service to respond to the acquisition request, determine the target traffic allocation quota of the traffic control device based on the total traffic allocation quota of the service cluster and the number of historical access requests corresponding to each of the flow control devices, and return it; wherein, the total traffic allocation quota is configured to indicate the number of access requests that the service cluster can support, and the target traffic allocation quota is configured to indicate the maximum number of access requests that can be responded to within a unit of time; a receiving module 5532, configured to receive the target traffic allocation quota returned based on the acquisition request; and a determining module 5533, configured to determine that the access operation corresponding to the target access request will be executed when the number of received target access requests does not reach the target traffic allocation quota.

[0194] Applying the above embodiments of this application, the traffic management service reasonably allocates a target traffic allocation quota (indicating the maximum number of access requests allowed to be responded to within a unit of time) to the business service cluster based on the total traffic allocation quota of the business service cluster (used to indicate the number of access requests that the business service cluster can support). The traffic control device controlling access requests entering the business service cluster then performs traffic control on the access requests entering the business service cluster according to the allocated target traffic allocation quota. That is, when the traffic control device determines that the number of received access requests has not reached the target traffic allocation quota, it determines that the access operation corresponding to the target access request will be executed, i.e., the target access request is allowed to access the business service cluster. Based on this, the stability of the business service cluster is improved.

[0195] In this way, by controlling the ingress traffic of the business service cluster, overall traffic control of the business service cluster can be achieved, improving the utilization rate of service computing resources and enhancing the rationality and efficiency of traffic control. At the same time, it reduces the occurrence of service anomalies due to excessive instantaneous load, thus improving the effectiveness of traffic control.

[0196] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the flow control method provided in this application.

[0197] This application also provides a computer-readable storage medium storing executable instructions, which, when executed by a processor, will cause the processor to execute the flow control method provided in this application.

[0198] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EP ROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0199] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0200] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0201] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0202] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A flow control system, characterized in that, The system includes: a traffic management service, and at least one traffic control device corresponding to the business service cluster; wherein, The traffic control device is used to send a target traffic allocation quota acquisition request to the traffic management service when it receives a target access request for the business service cluster. The target traffic allocation quota is used to indicate the maximum number of access requests that can be responded to within a unit of time. The traffic management service is used to obtain the total traffic allocation limit of the business service cluster, and the total traffic allocation limit is used to indicate the number of access requests that the business service cluster can support in responding to. In response to the acquisition request, based on the total traffic allocation quota and the number of historical access requests corresponding to each of the traffic control devices, the target traffic allocation quota of the traffic control device is determined, and the target traffic allocation quota is sent to the traffic control device. The flow control device is further configured to determine that when the number of received target access requests does not reach the target flow allocation quota, the access operation corresponding to the target access request will be executed. The flow control device is further configured to determine at least one target service corresponding to the target access request when the number of received target access requests does not reach the target flow allocation quota; and to send a first request for obtaining the service flow allocation quota to the flow management service for each target service, wherein the service flow allocation quota is used to indicate the maximum number of access requests allowed to respond to the target service within a unit of time. The traffic management service is further configured to, in response to the first acquisition request, determine the traffic allocation quota of the traffic control device for each target service based on the total traffic allocation quota of the target service and the number of historical access requests for the target service corresponding to each traffic control device; and send the traffic allocation quota to the traffic control device, wherein the total traffic allocation quota is used to indicate the number of access requests required by the target service. The traffic control device is further configured to, for each of the target services, determine that when the number of first target access requests corresponding to the target service in the target access requests does not reach the service traffic allocation quota, the access operation corresponding to the first target access request will be executed.

2. The system as described in claim 1, characterized in that, The traffic management service is also used to determine the total number of historical access requests corresponding to each of the traffic control devices in response to the acquisition request; Determine the ratio of the number of historical access requests corresponding to the flow control device to the total number of requests; The product of the total traffic allocation amount of the business service cluster and the ratio is used as the target traffic allocation amount of the traffic control device.

3. The system as described in claim 1, characterized in that, The traffic management service is also used to respond to the acquisition request and acquire the number of traffic control devices when the number of historical access requests corresponding to each traffic control device is zero. The total traffic allocation is averaged according to the number of devices to obtain the processing result; The processing result is used as the target flow allocation quota for the flow control device.

4. The system as described in claim 1, characterized in that, The flow control device is further configured to, when receiving a target access request for the service cluster, obtain the first sending time point corresponding to the last sending of the access request and the sending period of the access request; When the arrival of the second sending time point is determined based on the first sending time point and the sending period, a request to obtain the target traffic allocation quota is sent to the traffic management service.

5. The system as described in claim 1, characterized in that, The flow control device is also used to obtain a threshold for the number of request responses corresponding to the flow control device when a target access request for the business service cluster is received. When the number of received target access requests does not reach the request response number threshold, a request to obtain the target traffic allocation quota is sent to the traffic management service.

6. The system as described in claim 1, characterized in that, The traffic control device is also used to obtain the access quality score corresponding to the business service cluster when the number of received target access requests does not reach the target traffic allocation quota; When the access quality score is greater than the first quality score threshold, it is determined that the access operation corresponding to the target access request will be executed.

7. The system as described in claim 6, characterized in that, The flow control device is further configured to determine that an access operation corresponding to a portion of the target access request will be executed when the access quality score is less than the first quality score threshold and the access quality score is greater than the second quality score threshold. When the access quality score is less than the second quality score threshold, it is determined that the access operation corresponding to the target access request will be restricted.

8. The system as described in claim 1, characterized in that, The business service cluster includes multiple business services, at least two of which are used to form a service call chain, and the service call chain is used to implement the corresponding business; the system also includes a cluster traffic control service corresponding to each of the business services. The cluster traffic control service is used to obtain the request response quantity threshold corresponding to the first business service when the traffic control device controls the target access request to enter the business service cluster. When the number of target access requests passing through the first business service does not reach the request response number threshold, it is determined that the access operation of the first business service corresponding to the target access request will be executed. When the number of target access requests passing through the first business service reaches the request response number threshold, it is determined that the access operation of the first business service corresponding to the target access request will be restricted.

9. The system as described in claim 8, characterized in that, The cluster traffic control service is also used to, after restricting the access operation of the first service corresponding to the target access request, report the traffic overload notification message of the first service to the traffic control device through the service call chain; The flow control device is also used to receive the flow overload notification message; It has been determined that access requests to the service corresponding to the first service will be restricted.

10. The system as described in claim 8, characterized in that, The cluster traffic control service corresponding to the second business service is also used to determine the third business service that is next to the second business service in the service call chain, where the second business service is a non-last business service in the service call chain. Obtain the service quality score of the third business service; When the service quality score is lower than the quality score threshold, a service exception notification message for the third business service is reported to the traffic control device through the service call chain. The flow control device is also used to receive the service exception notification message; It has been determined that access requests to the services corresponding to the third service will be restricted.

11. The system as claimed in claim 1, characterized in that, The flow control device is further configured to determine at least one target object corresponding to the first target access request when the number of first target access requests corresponding to the target service does not reach the service flow allocation quota; For each of the target objects, a second acquisition request is sent to the traffic management service. The second acquisition request is used to acquire the number of historical access requests that have been responded to for the target object. The traffic management service is also used to obtain the object traffic allocation quota of the target object, and the object traffic allocation quota is used to indicate the maximum number of access requests allowed to be responded to for the target object; In response to the second acquisition request, for each target object, the number of historical access requests that have been responded to is acquired, and the number of responses is sent to the flow control device. The flow control device is further configured to, for each of the target objects, determine that when the number of second target access requests corresponding to the target object in the first target access request has not reached the number of responses, to execute the access operation corresponding to the second target access request.

12. The system as claimed in claim 1, characterized in that, The flow control device is also used to report the number of received access requests to the flow management service according to the reporting cycle; The traffic management service is also used to respond to the acquisition request and determine the target time period within the previous reporting cycle of the time point of the target access request. Obtain the number of historical access requests reported by each of the traffic control devices within the target time period; The number of reports corresponding to each of the aforementioned flow control devices shall be used as the number of historical access requests corresponding to the respective flow control device.

13. A flow control method, characterized in that, A traffic control system applied to at least one traffic control device, including a traffic management service and a corresponding business service cluster; the method includes: When the traffic control device receives a target access request for the business service cluster, it sends a request to obtain the target traffic allocation quota to the traffic management service. The acquisition request is used for the traffic management service to respond to the acquisition request, determine the target traffic allocation quota of the traffic control device based on the total traffic allocation quota of the business service cluster and the number of historical access requests corresponding to each traffic control device, and return it. Wherein, the total traffic allocation quota is used to indicate the number of access requests that the business service cluster can support, and the target traffic allocation quota is used to indicate the maximum number of access requests that can be responded to within a unit of time. Receive the target traffic allocation quota returned based on the acquisition request; When the number of the target access requests received does not reach the target traffic allocation quota, it is determined that the access operation corresponding to the target access request will be executed. When the number of received target access requests does not reach the target traffic allocation quota, at least one target service corresponding to the target access request is determined; for each target service, a first request to obtain the service traffic allocation quota is sent to the traffic management service, wherein the service traffic allocation quota is used to indicate the maximum number of access requests allowed to be responded to for the target service within a unit of time. The traffic management service receives the service traffic allocation quota returned by the traffic management service based on the first acquisition request. The service traffic allocation quota is determined by the traffic management service for each target service, the total service traffic allocation quota based on the target service, and the number of historical access requests for the target service corresponding to each traffic control device. The total service traffic allocation quota is used to indicate the number of access requests required by the target service. For each of the target services, if the number of first target access requests corresponding to the target service in the target access requests does not reach the service traffic allocation quota, it is determined that the access operation corresponding to the first target access request will be executed.

14. A flow control device, characterized in that, A traffic control system applied to at least one traffic control device, including a traffic management service and a corresponding business service cluster; the device includes: The sending module is used to send a request to obtain the target traffic allocation quota to the traffic management service when it receives a target access request for the business service cluster. The acquisition request is used for the traffic management service to respond to the acquisition request, determine the target traffic allocation quota of the traffic control device based on the total traffic allocation quota of the business service cluster and the number of historical access requests corresponding to each traffic control device, and return it. Wherein, the total traffic allocation quota is used to indicate the number of access requests that the business service cluster can support, and the target traffic allocation quota is used to indicate the maximum number of access requests that can be responded to within a unit of time. The receiving module is used to receive the target traffic allocation quota returned based on the acquisition request; The determination module is used to determine that when the number of received target access requests does not reach the target traffic allocation quota, the access operation corresponding to the target access request will be executed. The sending module is further configured to determine at least one target service corresponding to the target access request when the number of received target access requests does not reach the target traffic allocation quota; and to send a first request for obtaining the traffic allocation quota to the traffic management service for each target service, wherein the traffic allocation quota is used to indicate the maximum number of access requests allowed to be responded to for the target service within a unit of time. The receiving module is further configured to receive the service traffic allocation quota returned by the traffic management service based on the first acquisition request. The service traffic allocation quota is determined by the traffic management service for each target service, the total service traffic allocation quota based on the target service, and the number of historical access requests for the target service corresponding to each traffic control device. The total service traffic allocation quota is used to indicate the number of access requests required by the target service. The determining module is further configured to, for each of the target services, determine that when the number of first target access requests corresponding to the target service in the target access requests does not reach the service traffic allocation quota, the access operation corresponding to the first target access request will be executed.

15. An electronic device, characterized in that, The electronic device includes: Memory, used to store executable instructions; The processor, when used with executable instructions stored in the memory, implements the flow control method of claim 13.

16. A computer-readable storage medium storing executable instructions, characterized in that, When the executable instructions are executed by the processor, they implement the flow control method of claim 13.

17. A computer program product comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, the flow control method of claim 13 is implemented.

Images