Flow limiting method and system based on distributed cache strategy

Abstract

The invention provides a flow limiting method and system based on a distributed cache strategy, belongs to the technical field of distributed flow limiting, and is applied to a distributed cache system which comprises a plurality of cache partitions. The method comprises the following steps: acquiring real-time traffic and historical traffic of each cache partition; inputting the real-time traffic and the historical traffic into a traffic prediction model to obtain predicted traffic at the next moment; and determining a flow limiting parameter of the cache partition based on the predicted flow, and limiting the flow at the next moment based on the flow limiting parameter. According to the method, the flow limiting parameters are automatically adjusted by using the machine learning technology, the data flow can be accurately controlled, the system overload is prevented, the resource waste caused by excessive limitation is avoided, and the flow management efficiency and effect of the distributed cache system are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of distributed current limiting, and in particular to a current limiting method and system based on a distributed cache strategy. Background Art

[0002] In today's society where digitalization and networking are developing rapidly, data traffic management has become a major challenge for network service providers and enterprises. Especially in the fields of cloud services, video streaming, online games and e-commerce, how to effectively manage and control traffic to avoid network congestion and service degradation has become a key factor in maintaining competitiveness and user satisfaction.

[0003] The current limiting methods in the existing technology are mainly concentrated in two categories: single-node current limiting and centralized current limiting. Single-node current limiting is usually implemented at the local server or application level, such as using a leaky bucket algorithm or a token bucket algorithm to control the request rate entering the system. Centralized current limiting strategies are managed on a central server or a dedicated current limiting service to uniformly schedule the request processing capacity of the entire network or system. There are still some shortcomings in distributed and highly dynamic network environments: (1) Lack of flexibility and scalability: Single-node and centralized current limiting technologies are difficult to flexibly cope with large-scale or geographically distributed network environments. These technologies have certain limitations in scenarios where the current limiting strategy needs to be dynamically adjusted to respond to real-time network status changes. (2) Single point of failure problem: In a centralized current limiting solution, all decision-making and data processing rely on the central node. Once the central node fails, the current limiting function of the entire system may fail directly. (3) Processing delay: When processing cross-regional traffic, the centralized system may not be able to respond to traffic changes in real time due to data transmission and processing delays, thereby affecting service quality and user experience.

[0004] Therefore, there is an urgent need to provide a flow limiting method and system based on a distributed caching strategy to achieve accurate flow limiting of large-scale and highly variable network traffic. Summary of the invention

[0005] In view of this, it is necessary to provide a current limiting method and system based on a distributed caching strategy to solve the technical problem that single-node current limiting and centralized current limiting in the prior art have poor current limiting effects when facing large-scale and highly variable network traffic.

[0006] On the one hand, in order to solve the above technical problems, the present invention provides a current limiting method based on a distributed cache strategy, which is applied to a distributed cache system, wherein the distributed cache system includes multiple cache partitions; the method includes: Obtaining real-time traffic and historical traffic of each cache partition; Inputting the real-time traffic and the historical traffic into a traffic prediction model to obtain the predicted traffic at the next moment; A flow limiting parameter of the cache partition is determined based on the predicted flow, and flow limiting is performed on the flow at the next moment based on the flow limiting parameter.

[0007] In a possible implementation, determining the flow limiting parameter of the cache partition based on the predicted flow includes: Configuring a flow class handle of the cache partition based on a flow controller, wherein the flow class handle includes a preset flow rate configuration; The preset flow rate configuration is adjusted based on the predicted flow to obtain the flow limiting parameter.

[0008] In a possible implementation, the method further includes: The business importance and service level agreement of the real-time traffic are obtained, and the business importance and service level agreement are input into a priority determination model to obtain traffic priority.

[0009] In a possible implementation, the method further includes: Obtaining the response time of the request to be responded to, the throughput of the distributed cache system and the user satisfaction; Determining whether the traffic prediction model needs to be retrained based on the response time, the throughput, and the user satisfaction; If so, the traffic prediction model is retrained based on the historical traffic.

[0010] In a possible implementation manner, before determining the flow limiting parameter of the cache partition based on the predicted flow, the method further includes: Obtaining a request type and an expected access frequency of the request to be responded to, and determining partition parameters of the plurality of cache partitions based on the request type and the expected access frequency; The partition parameters are optimized based on the real-time traffic and the system load of the distributed cache system to obtain optimized partition parameters.

[0011] In a possible implementation, the optimizing the partition parameters based on the real-time traffic and the system load to obtain the optimized parameters includes: Determining a required load of the cache partition based on the real-time traffic and the system load; Obtaining a load threshold of the cache partition, and determining whether the cache partition needs to be adjusted based on the load threshold and the required load; If yes, the partition parameters are optimized based on a load balancing algorithm to obtain the optimized parameters.

[0012] In a possible implementation, the method further includes: Obtaining a cache hit rate and a response time of the cache partition, and determining a first cache evaluation result based on the cache hit rate and the response time; Obtaining user feedback and behavior analysis, and determining a second cache evaluation result based on the user feedback and the behavior analysis; Based on the first cache evaluation result and the second cache evaluation result, determine whether a partition parameter needs to be adjusted; if so, adjust the partition parameter.

[0013] In a possible implementation, the distributed cache system further includes a plurality of data centers, and the corresponding relationship between the data centers and the cache partitions is one-to-many, and the method further includes: Get the source IP address and user ID of the request to be responded to; Determine the geographic location of the request to be responded to based on the source IP address and the user ID; A target data center among the plurality of data centers is determined based on the geographic location.

[0014] In a possible implementation, the cache partition includes a local cache and a distributed cache; The read strategy of the cache partition is: read data from the local cache first, and return to the distributed cache when there is a miss; the write strategy of the cache partition is: update strategy when writing.

[0015] The present invention also provides a current limiting system based on a distributed cache strategy, which is applied to a distributed cache system, wherein the distributed cache system includes a plurality of cache partitions; the current limiting system based on a distributed cache strategy includes: A cache partition flow acquisition unit, used to acquire the real-time flow and historical flow of each cache partition; A traffic prediction unit, used for inputting the real-time traffic and the historical traffic into a traffic prediction model to obtain the predicted traffic at the next moment; A flow limiting unit is used to determine the flow limiting parameters of the cache partition based on the predicted flow, and to limit the flow at the next moment based on the flow limiting parameters.

[0016] The beneficial effects of the present invention are as follows: the flow limiting method based on the distributed cache strategy provided by the present invention obtains the real-time flow and historical flow of the cache partition, and determines the predicted flow at the next moment based on the real-time flow and historical flow, and predicts the flow of the rapidly changing network environment through the flow prediction model, thereby obtaining accurate predicted flow, and then the flow limiting parameters of the cache partition can be determined based on the predicted flow, thereby achieving accurate flow limiting of the flow at the next moment, which can significantly improve the flow management efficiency and effect of the distributed cache system, and provide more flexible and efficient support for modern network services. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 A schematic flow chart of an embodiment of a current limiting method based on a distributed cache strategy provided by the present invention; Figure 2 For the present invention Figure 1 A schematic diagram of an embodiment of a flow chart of determining a current limiting parameter of a cache partition in step S103; Figure 3 A schematic diagram of a flow chart of an embodiment of optimizing a flow prediction model provided by the present invention; Figure 4 A schematic diagram of a flow chart of an embodiment of setting and optimizing partition parameters of a cache partition provided by the present invention; Figure 5 For the present invention Figure 4 A schematic flow chart of an embodiment of step S402; Figure 6 A schematic diagram of a flow chart of an embodiment of further optimizing partition parameters provided by the present invention; Figure 7 A schematic diagram of a process flow of an embodiment of determining a target data center provided by the present invention; Figure 8 A schematic diagram of the structure of an embodiment of a current limiting system based on a distributed cache strategy provided by the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0020] It should be understood that the schematic drawings are not drawn to scale. The flowchart used in the present invention shows the operations implemented according to some embodiments of the present invention. It should be understood that the operations of the flowchart can be implemented out of order, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art, under the guidance of the content of the present invention, can add one or more other operations to the flowchart, and can also remove one or more operations from the flowchart. Some of the block diagrams shown in the accompanying drawings are functional entities, which do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor systems and / or microcontroller systems.

[0021] Reference to an "embodiment" herein means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0022] The present invention provides a current limiting method and system based on a distributed cache strategy, which is described below.

[0023] Figure 1 A schematic flow chart of an embodiment of a current limiting method based on a distributed cache strategy provided by the present invention is applied to a distributed cache system, where the distributed cache system includes multiple cache partitions; Figure 1 As shown in the figure, the current limiting methods based on the distributed cache strategy include: S101, obtaining the real-time flow and historical flow of each cache partition; S102, inputting the real-time traffic and historical traffic into the traffic prediction model to obtain the predicted traffic at the next moment; S103: Determine a flow limiting parameter of the cache partition based on the predicted flow, and limit the flow at the next moment based on the flow limiting parameter.

[0024] The method for acquiring real-time traffic in step S101 is as follows: deploy a dedicated monitoring agent in each cache partition, and accurately record the arrival time, processing time, source and target information of each request to be responded to based on the monitoring agent, so as to obtain real-time traffic.

[0025] Specifically, the monitoring agent is deployed through Prometheus.

[0026] The method for obtaining the historical traffic in step S102 is: obtaining the historical traffic from a data warehouse storing historical traffic data. The data warehouse may be Apache Hadoop, which supports subsequent large-scale data storage and complex queries.

[0027] Since the traffic prediction model is a time series prediction, in a preferred embodiment of the present invention, the model structure of the traffic prediction model is a Long Short-Term Memory (LSTM) network.

[0028] It should be understood that: before step S102, the constructed initial traffic prediction model needs to be trained, tested and verified to obtain a traffic prediction model for predicting traffic.

[0029] Among them, before training, testing and verifying the initial traffic prediction model, it is necessary to first build a data set, and then perform training, testing and verification based on the data set.

[0030] Specifically, a data set is created using a sliding window method, which includes multiple input (feature) and output (label) pairs. The input includes time, request volume, and response time.

[0031] Before using the dataset, it is necessary to preprocess the dataset, which includes but is not limited to data quality check, data conversion, and standardization.

[0032] Data quality detection includes missing value checking and outlier processing. Missing value checking is to ensure that there are no missing data points in the time series. If there are, you can choose to fill (for example, use the previous value, mean, or median) or delete the missing records. Outlier processing is to identify and handle outliers, which may be caused by measurement errors or other atypical events and may have an adverse effect on model training.

[0033] Data transformation and standardization are as follows: 1. It is beneficial to standardize or normalize the data to a uniform scale. For example, Min-Max scaling or Z-score standardization can be used. 2. Encoding: If the data contains categorical features (such as the day of the week, whether it is a holiday, etc.), these features need to be converted into a format that the model can process through one-hot encoding or label encoding.

[0034] It should be noted that the time span of historical traffic can be set according to the actual application scenario, such as 12 hours or 24 hours, etc., and is not specifically limited here.

[0035] To improve the accuracy of traffic prediction, in some embodiments of the present invention, additional time features may be added to real-time traffic and historical traffic, such as weekdays, weekends, or statutory holidays, to help the model better learn the impact of time changes. Specifically, real-time traffic and historical traffic may also include traffic trend features.

[0036] In some embodiments of the present invention, the specific process of training, testing and verifying the constructed initial traffic prediction model is as follows: Data segmentation: Ensure that the data set is arranged in chronological order. For time series data, the most important thing is to maintain time continuity to avoid prediction bias caused by disordered data timing.

[0037] Data division: Divide the data according to the amount of data, select 20%~30% of the data as the test set, and the rest as the training set. For example, for the data of 2023, select the data of the first 9 months to train the model, and the data of the last 3 months to test.

[0038] Model configuration: Configure the LSTM model structure.

[0039] By adjusting the following parameters: Number of layers and neurons: In this example, 50 to 100 neurons are set, which can be increased or decreased according to the number of features and resource size.

[0040] Learning rate: Set to 0.001. If the performance is unstable during training, lower it appropriately.

[0041] Batch size: The number of samples involved in each batch, which is directly related to memory usage and training speed. The default setting is 32, which can be adjusted appropriately according to the actual GPU computing power. If the computing power is sufficient, 64 or 128 can be selected. Find the balance between speed and performance through specific experiments. Refer to the following specifications when making a selection: Optimizer: The Adam optimizer is selected. It has good algorithm optimization performance for simple prediction scenarios and supports automatic adjustment of learning rate, which is suitable for the current prediction model LSTM.

[0042] The specific training process is as follows: After the above configuration, follow the following process to load preprocessed data, build compiled model, train, and save pretrained weights.

[0043] Set Epochs to 50 and use the historical dataset for 50 rounds of training. Adjust the number of Epochs to 50-100 based on the validation results.

[0044] Loss function: The mean squared error (MSE) function is specified when compiling the model. The smaller the loss value, the closer it is to the truth and the better the prediction performance.

[0045] Early stopping function: When the loss value does not decrease or increase after multiple epochs, training needs to be terminated early to avoid resource waste and prevent model overfitting.

[0046] Training correction: When the loss value does not show a good downward trend or early stopping occurs, you can increase or decrease the number of LSTM layers, adjust the number of neurons in each layer, switch optimizers, reduce the learning rate, increase batch_size, and increase the number of epochs. If the parameters do not work well, you need to consider adding more easily identifiable feature data to the training set.

[0047] The specific process of model validation is as follows: The trained model is evaluated by the mean square error (MSE) and root mean square error (RMSE). The smaller the calculated values ​​of the two, the better the evaluation performance.

[0048] The specific process of model testing is as follows: 1. Compare the performance differences on the training set and the test set. If the error on the test set is much higher than that on the training set, there may be overfitting.

[0049] 2. Use cross-validation and other methods to further confirm the stability and reliability of the model.

[0050] 3. Result visualization: Draw a comparison chart of actual values ​​and predicted values ​​to intuitively demonstrate the accuracy of the model's prediction and its ability to grasp trends. Analyze the distribution of prediction errors and check whether there are systematic deviations.

[0051] It should be noted that the prediction interval of the predicted traffic can be set or adjusted according to parameters such as the access speed of the request to be responded to, for example, the traffic prediction model automatically runs every five minutes.

[0052] Compared with the prior art, the flow limiting method based on the distributed cache strategy provided in the embodiment of the present invention obtains the real-time flow and historical flow of the cache partition, and determines the predicted flow at the next moment based on the real-time flow and historical flow. The flow of the rapidly changing network environment is predicted by the flow prediction model, and accurate predicted flow is obtained. Then, the flow limiting parameters of the cache partition can be determined based on the predicted flow, and accurate flow limiting of the flow at the next moment can be achieved. This can significantly improve the flow management efficiency and effect of the distributed cache system, and provide more flexible and efficient support for modern network services.

[0053] In some embodiments of the present invention, Figure 2 As shown, the flow limiting parameters of the cache partition based on the predicted flow in step S103 include: S201, configuring a traffic class handle of a cache partition based on a traffic controller (Traffic Control, TC), wherein the traffic class handle includes a preset flow rate configuration; S202: Adjust the preset flow rate configuration based on the predicted flow rate to obtain flow limiting parameters.

[0054] The traffic class handle of the cache partition configured based on the TC in step S201 is specifically: Define queue rules and create queues based on them; Define class rules, which include multiple root nodes, eth0 devices, and their corresponding traffic class handles, and preset traffic configuration for each traffic class handle; Define application filters and add filter rules. The filter mounts the traffic class handle. The filter rules define the correspondence between the traffic class handle and each port.

[0055] The pseudo code of the above process is as follows: “# Define queue discipline # Set up a bandwidth allocation system based on htb (hierarchical token bucket) # qdisc add Create a queue # dev eth0 specifies the network card interface eth0 # root hanlde 1: Set the queue rules and the handle identifier is 1 # default 10 specifies the default class for unclassified traffic tc qdisc add dev eth0 root handle 1: htb default 10 # Define class rules # Create a traffic class handle of 1:1 under the root node (parent node 1) and specify a bandwidth of 50Mbps tc class add dev eth0 parent 1: classid 1:1 htb rate 50mbit # Create a traffic class handle of 1:2 under the root node (parent node 1) and specify a bandwidth of 20Mbps tc class add dev eth0 parent 1: classid 1:2 htb rate 20mbit # Change the bandwidth limit of class 1:1 on device eth0 from 50Mbps to 100Mbps tc class change dev eth0 parent 1: classid 1:1 htb rate 100mbit # Define application filters # Add filter rules # filter add Add a new filter # dev eth0 distributes traffic on network interface eth0 # protocol ip specifies the protocol as ip. You can also use ipv6 or other network layer protocols # parent 1:0 refers to the filter mount point handle, associated with the root qdisc # prio 1 The filter priority is 1, arranged as needed # u32 Common and flexible filter matching mechanism # match ip dst 192.168.1.100 / 32 matches the destination IP address 192.168.1.100. You can also select src (source address), dport (destination port), sport (source port), icmp type (ICMP message type), etc. The mask used (such as 0xffff) can be changed to support partial matching.

[0056] # flowid 1:1 directs traffic to the class with handle 1:1 tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ip dst192.168.1.100 / 32 flowid 1:1 # Direct all traffic destined for port 80 to the class with handle 1:2 under the qdisc queue.

[0057] tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ipdport 80 0xffff flowid 1:2”.

[0058] In some embodiments of the present invention, the flow limiting parameters include a Quality of Service (QoS) level and a request queuing strategy.

[0059] Among them, QoS describes the service quality of the network, including various technical indicators such as delay, packet loss rate, throughput, etc., and is often used in network traffic management and priority setting.

[0060] In a specific embodiment of the present invention, step S202 is specifically: Get the high threshold flow index and the low threshold flow index. If the predicted flow is greater than the high threshold flow index, switch the preset flow rate configuration to the corresponding low flow rate index. For example, the flow rate index exceeding 1000 but less than 2000 is defined as 60mbit. Here, the predicted flow is 1200, so it is switched to 60mbit. If the predicted flow is less than the low threshold flow index, switch to the flow rate index corresponding to the low-speed scenario.

[0061] It should be noted that steps S201 - S202 may be automated using cron jobs.

[0062] Since the traffic of financial transactions and other services is of high importance, in order to ensure the normal operation of this service, it is necessary to ensure that the traffic of this service is processed preferentially. Therefore, in some embodiments of the present invention, the current limiting method based on the distributed cache strategy also includes: The business importance and service level agreement of real-time traffic are obtained and input into a prioritization model to obtain traffic priority.

[0063] The embodiment of the present invention can ensure priority response to important business traffic by performing differentiated processing on traffic and prioritizing real-time traffic.

[0064] It should be noted that the process of determining the flow limiting parameters in step S103 can also take traffic priority into consideration, that is, the flow limiting parameters of the cache partition are determined based on the predicted traffic and traffic priority to further improve the accuracy of the flow limiting parameters and their adaptability to business scenarios.

[0065] Since the priorities of different services may change over time, in some embodiments of the present invention, the traffic priority may be dynamically adjusted based on real-time service requirements and user behavior to further improve the real-time performance of the traffic priority.

[0066] Specifically, the service priority of paying users can be increased when resources are scarce.

[0067] To ensure the accuracy of the predicted flow, in some embodiments of the present invention, Figure 3 As shown, the current limiting method based on the distributed cache strategy also includes: S301, obtaining the response time of the request to be responded to, the throughput of the distributed cache system and the user satisfaction; S302, judging whether the traffic prediction model needs to be retrained based on response time, throughput and user satisfaction; S303: If yes, retrain the traffic prediction model based on historical traffic.

[0068] The retraining of the traffic prediction model in step S303 includes but is not limited to: adjusting model parameters and replacing model algorithms.

[0069] The embodiment of the present invention can adjust and update the traffic prediction model to adapt to changes in the network environment and new business needs by judging whether the traffic prediction model needs to be retrained based on response time, throughput and user satisfaction. The flow limiting parameters determined based on the traffic prediction model can meet the response time, throughput and user satisfaction requirements during use, thereby improving the rationality and accuracy of the flow limiting parameters. By retraining the traffic prediction model, the embodiment of the present invention can maximize the processing power and resource utilization of the distributed cache system while maintaining the quality of network services, and has important adaptability and value for dynamic and changeable network environments.

[0070] Since the predicted traffic needs to be written into the cache partition, or read from the cache partition, in order to make the response of reading and writing the predicted traffic faster, in some embodiments of the present invention, Figure 4 As shown, before step S103, it also includes: S401, obtaining a request type and an expected access frequency of a request to be responded to, and determining partition parameters of multiple cache partitions based on the request type and the expected access frequency; S402: Optimize partition parameters based on real-time traffic and system load of the distributed cache system to obtain optimized partition parameters.

[0071] The request type can be a static request type or a dynamic request type. The dynamic request type is a request with frequent data updates, and the static request type is a request with infrequent data updates. The static request type is more suitable for long-term caching.

[0072] The expected access frequency can be divided into high-frequency access and low-frequency access based on the frequency threshold. High-frequency access, such as popular video clips, needs to be retained in the cache for a longer time, and low-frequency access, such as API responses, has a shorter life cycle in the cache.

[0073] The partition parameters include but are not limited to cache size and data expiration policy.

[0074] The embodiment of the present invention determines the partition parameters of multiple cache partitions based on the obtained request type and expected access frequency, allowing the distributed cache system to adjust resource allocation according to the request type and expected access rate, thereby optimizing the response rate of requests to be responded to and improving the efficiency of data writing and reading in the cache partitions.

[0075] Furthermore, the embodiments of the present invention are configured to optimize partition parameters based on real-time traffic and system load, and can dynamically expand or shrink partition parameters to ensure that the distributed cache system can continue to operate stably under load fluctuations during peak traffic periods, thereby ensuring optimization of system performance and resource utilization.

[0076] In some embodiments of the present invention, Figure 5 As shown, step S402 includes: S501, determining the required load of the cache partition based on the real-time traffic and system load; S502: Obtain a load threshold of a cache partition, and determine whether the cache partition needs to be adjusted based on the load threshold and the required load; S503: If yes, optimize the partition parameters based on the load balancing algorithm to obtain optimized parameters.

[0077] The demand load in step S501 is the difference between the real-time flow and the system load.

[0078] Among them, the load threshold in step S502 includes a minimum load and a maximum load, and the cache partition is adjusted specifically as follows: when the required load is less than the minimum load, the cache partition is shrunk; when the required load is greater than the maximum load, the cache partition is expanded.

[0079] The contraction process specifically includes: reducing the cache size of the cache partition or triggering data migration; the expansion process specifically includes: increasing the cache size of the cache partition or triggering data migration.

[0080] The embodiment of the present invention can improve the resource utilization of the load by determining the required load and dynamically determining whether the partition parameters need to be adjusted based on the required load. In addition, when optimizing the partition parameters, based on the load balancing algorithm optimization, the load balancing algorithm can be used to redistribute the cache across nodes to ensure the balance of the load of each node and further optimize the resource utilization efficiency.

[0081] In a specific embodiment of the present invention, in order to ensure the consistency of cache partition data, a consistent hashing algorithm is applied to process the data distribution between cache partition nodes to ensure that the need for data redistribution is reduced when adding and removing cache nodes, thereby maintaining the stability of the cache partition.

[0082] Specifically, the hash ring technology is used to distribute data in the Redis cluster to ensure uniform data distribution and optimize load balancing. When adding or removing nodes, the nature of the consistent hashing algorithm minimizes data migration and only affects a small part of the data on the hash ring, such as implemented in the Apache Cassandra database.

[0083] Furthermore, to avoid technical issues such as data loss caused by failures during cache partition allocation, critical data is replicated in multiple physical locations to ensure that services can be quickly restored in the event of any cache partition failure.

[0084] Specifically, Write-Ahead Logging (WAL): Before data is changed, a log is written first, using a WAL mechanism such as the PostgreSQL database to ensure that any database transaction is recorded before it is committed, so that these transactions can be recovered after a system crash.

[0085] Furthermore, when a distributed cache system fails, an automatic fault recovery mechanism can be set up. Specifically, for temporary faults, such as network delays or temporary service non-response, the system will automatically retry the request. This operation is implemented by configuring the retry policy in the Nginx reverse proxy server or service mesh such as Istio. For persistent faults, the system automatically switches to a hot standby server or service, such as using AWS Route 53 for DNS failover to ensure the continuity of key services.

[0086] Since the network environment is changing dynamically and the service requirements are also changing dynamically, in order to realize the adaptability of the cache partition to the dynamically changing network environment and service requirements, in some embodiments of the present invention, for example Figure 6 As shown, the current limiting method based on the distributed cache strategy also includes: S601, obtaining a cache hit rate and a response time of a cache partition, and determining a first cache evaluation result based on the cache hit rate and the response time; S602: Obtain user feedback and behavior analysis, and determine a second cache evaluation result based on the user feedback and behavior analysis; S603: Determine whether partition parameters need to be adjusted based on the first cache evaluation result and the second cache evaluation result; if so, adjust the partition parameters.

[0087] The embodiment of the present invention determines whether the partition parameters need to be adjusted based on the first cache evaluation result and the second cache evaluation result, and can judge whether the partition parameters meet the dynamically changing network parameters and business requirements. If they do not meet the requirements, the partition parameters are adjusted to ensure that the partition parameters adapt to changes in business requirements and changes in the network environment.

[0088] In a specific embodiment of the present invention, the distributed cache system further includes multiple data centers, and the corresponding relationship between the data centers and the cache partitions is one-to-many, that is, multiple cache partitions and a data center are located in the same geographical location range.

[0089] Based on the above distributed cache system, in order to further improve the response speed of the cache partition to the request to be responded, in some embodiments of the present invention, such as Figure 7 As shown, the current limiting method based on the distributed cache strategy also includes: S701, obtaining the source IP address and user ID of the request to be responded to; S702, determining the geographic location of the request to be responded to based on the source IP address and the user ID; S703: Determine a target data center among multiple data centers based on the geographical location.

[0090] The embodiment of the present invention first determines a target data center based on the geographical location of the request to be responded to, and then optimizes partition parameters of multiple cache partitions corresponding to the target data center.

[0091] The embodiment of the present invention determines the target data center based on the geographical location, thereby improving the level of refined management of traffic and effectively improving the efficiency of traffic management in the distributed cache system.

[0092] It should be noted that: in order to ensure the failure recovery rate, key data can be asynchronously replicated between different data centers, using mechanisms such as MySQL replication to implement master-slave replication to ensure data consistency and continuous availability.

[0093] To further improve the response rate of pending requests, in some embodiments of the present invention, the cache partition includes a local cache and a distributed cache; The read strategy of the cache partition is: read data from the local cache first, and return to the distributed cache if it misses; the write strategy of the cache partition is: update when writing (write-through) strategy.

[0094] The embodiment of the present invention reduces data access delay and improves data retrieval speed based on local cache by setting cache partitions including two-level local cache and distributed cache, and improves data availability and fault tolerance based on distributed cache.

[0095] The local cache uses memory cache technology, such as Redis or Memcached. Its configuration is to set an appropriate expiration policy, such as an automatic elimination mechanism based on the LRU algorithm.

[0096] Distributed cache is deployed in multiple geographical locations and uses consistent hashing technology to achieve load balancing. It can store multiple copies of data and improve the speed of fault recovery.

[0097] It should be noted that local cache and distributed cache need to be cleaned up regularly, and expired or infrequently accessed data needs to be checked and cleaned up regularly. At the same time, cache data at all levels needs to be synchronized to ensure consistency and the latest status of the data.

[0098] To ensure efficient operation and responsiveness of multi-level cache, a performance monitoring and optimization mechanism is added. The detailed implementation steps are as follows: 1) Monitoring system deployment Monitoring tool selection: Choose the right monitoring tool, such as Prometheus combined with Grafana to visualize monitoring data, or use commercial solutions such as New Relic and Datadog to provide comprehensive performance monitoring.

[0099] Performance indicator definition: Hit rate: A key indicator for measuring cache effectiveness. A high hit rate indicates that most requests are cached successfully, reducing the pressure on the backend database.

[0100] Latency: Monitor the response time of each cache request and the average time it takes for data to be read from the cache. This is key to responding to user requests quickly.

[0101] Capacity utilization: Track cache resource usage, including memory and disk space usage, to ensure that the cache tier has enough resources to handle the expected load.

[0102] Monitoring frequency and alarm threshold: Set the frequency of data collection and the alarm threshold when performance is lower than expected. When performance data is abnormal, the alarm can be triggered in time and the administrator will be notified.

[0103] 2) Performance Tuning Dynamic resource allocation: Dynamically adjust the size and quantity of cache based on real-time monitoring data. For example, if a cache node frequently reaches its capacity limit, the system can automatically expand the cache capacity or add nodes.

[0104] Configuration optimization: Optimize cache configuration parameters, such as adjusting the cache expiration policy or updating the data refresh frequency to adapt to changing usage patterns.

[0105] Automated optimization scripts: Develop automated scripts or use intelligent tuning tools, such as AWS Auto Scaling, to automatically adjust resource allocation according to load changes to ensure the optimal balance between performance and cost.

[0106] 3) Feedback mechanism User feedback collection: Collect user feedback on cache performance through user surveys, online feedback tools, or customer support services, especially after cache updates or configuration changes.

[0107] System log analysis: System logs provide a detailed record of cache operations. Analyzing these logs can help identify cache-level problems, such as frequent cache penetration or cache avalanche.

[0108] Continuous improvement process: Establish a mechanism for regular review and iterative improvement, incorporate user feedback and log analysis results into the continuous optimization of cache strategies and architectures, and ensure the continuous adaptability and performance improvement of the system.

[0109] In order to further improve the fault tolerance of the distributed cache system, in some embodiments of the present invention, 1) System redundancy design: Multiple instances of key components can be deployed, for example, using cluster configuration, such as: 1) Use Kubernetes clusters to manage multiple copies of microservices to ensure that the service can continue to run even if a single component fails.

[0110] 2) Fault isolation strategy: Use microservice architecture to separate different service units to ensure that when some system components fail, it will not affect the normal operation of other unrelated components. For example, use Docker containerization technology to deploy services, and each service runs in an independent container to isolate the spread of faults.

[0111] In summary, the current limiting method based on the distributed cache strategy provided by the embodiment of the present invention intelligently classifies data and allocates resources according to the attributes and sources of the traffic to ensure that all types of requests can be optimally processed. The current limiting parameters are automatically adjusted using machine learning technology. This adaptive mechanism can accurately control the data flow, prevent system overload, and avoid waste of resources caused by excessive restrictions. In addition, setting a multi-level cache strategy and an advanced fault-tolerant mechanism not only enhances the scalability and response speed of the system, but also improves the reliability and fault recovery capability of the system.

[0112] The embodiments of the present invention innovatively provide a flexible, reliable and efficient method to handle large-scale and highly variable network traffic, which can significantly improve the efficiency and effectiveness of traffic management in distributed systems, provide more stable, flexible and efficient support for modern network services, and are particularly suitable for the needs of modern distributed systems and cloud infrastructure. In addition, the implementation of this method also helps to reduce the overall maintenance cost and operational complexity of the system, providing network service providers and enterprises with a powerful tool to ensure the continuity and high availability of key services.

[0113] In order to better implement the current limiting method based on the distributed cache strategy in the embodiment of the present invention, on the basis of the current limiting method based on the distributed cache strategy, the embodiment of the present invention also provides a current limiting system based on the distributed cache strategy, which is applied to the distributed cache system, and the distributed cache system includes multiple cache partitions; Figure 8 As shown, the current limiting system 800 based on the distributed cache strategy includes: The cache partition traffic acquisition unit 801 is used to acquire the real-time traffic and historical traffic of each cache partition; The traffic prediction unit 802 is used to input the real-time traffic and historical traffic into the traffic prediction model to obtain the predicted traffic at the next moment; The flow limiting unit 803 is used to determine the flow limiting parameters of the cache partition based on the predicted flow, and to limit the flow at the next moment based on the flow limiting parameters.

[0114] The current limiting system 800 based on distributed cache strategy provided in the above embodiment can implement the technical solution described in the above embodiment of the current limiting method based on distributed cache strategy. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above embodiment of the current limiting method based on distributed cache strategy, which will not be repeated here.

[0115] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the computer program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0116] The above is a detailed introduction to a current limiting method and system based on a distributed caching strategy provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A current limiting method based on a distributed cache strategy, characterized in that: Applied to a distributed cache system, the distributed cache system includes multiple cache partitions; the method includes: Obtaining real-time traffic and historical traffic of each cache partition; Inputting the real-time traffic and the historical traffic into a traffic prediction model to obtain the predicted traffic at the next moment; A flow limiting parameter of the cache partition is determined based on the predicted flow, and flow limiting is performed on the flow at the next moment based on the flow limiting parameter.

2. The current limiting method based on the distributed cache strategy according to claim 1 is characterized in that: The determining the current limiting parameter of the cache partition based on the predicted traffic includes: Configuring a flow class handle of the cache partition based on a flow controller, wherein the flow class handle includes a preset flow rate configuration; The preset flow rate configuration is adjusted based on the predicted flow to obtain the flow limiting parameter.

3. The current limiting method based on the distributed cache strategy according to claim 1 is characterized in that: The method further comprises: The business importance and service level agreement of the real-time traffic are obtained, and the business importance and service level agreement are input into a priority determination model to obtain traffic priority.

4. The current limiting method based on the distributed cache strategy according to claim 1 is characterized in that: The method further comprises: Obtaining the response time of the request to be responded to, the throughput of the distributed cache system and the user satisfaction; Determining whether the traffic prediction model needs to be retrained based on the response time, the throughput, and the user satisfaction; If so, the traffic prediction model is retrained based on the historical traffic.

5. The current limiting method based on the distributed cache strategy according to claim 1 is characterized in that: Before determining the flow limiting parameter of the cache partition based on the predicted flow, the method further includes: Obtaining a request type and an expected access frequency of the request to be responded to, and determining partition parameters of the plurality of cache partitions based on the request type and the expected access frequency; The partition parameters are optimized based on the real-time traffic and the system load of the distributed cache system to obtain optimized partition parameters.

6. The current limiting method based on the distributed cache strategy according to claim 5 is characterized in that: The optimizing the partition parameters based on the real-time traffic and the system load to obtain the optimized parameters includes: Determining a required load of the cache partition based on the real-time traffic and the system load; Obtaining a load threshold of the cache partition, and determining whether the cache partition needs to be adjusted based on the load threshold and the required load; If yes, the partition parameters are optimized based on a load balancing algorithm to obtain the optimized parameters.

7. The current limiting method based on the distributed cache strategy according to claim 5 is characterized in that: The method further comprises: Obtaining a cache hit rate and a response time of the cache partition, and determining a first cache evaluation result based on the cache hit rate and the response time; Obtaining user feedback and behavior analysis, and determining a second cache evaluation result based on the user feedback and the behavior analysis; Based on the first cache evaluation result and the second cache evaluation result, determine whether a partition parameter needs to be adjusted; if so, adjust the partition parameter.

8. The current limiting method based on the distributed cache strategy according to any one of claims 1 to 7, characterized in that: The distributed cache system further includes a plurality of data centers, and the corresponding relationship between the data centers and the cache partitions is one-to-many, and the method further includes: Get the source IP address and user ID of the request to be responded to; Determine the geographic location of the request to be responded to based on the source IP address and the user ID; A target data center among the plurality of data centers is determined based on the geographic location.

9. The current limiting method based on the distributed cache strategy according to claim 1 is characterized in that: The cache partition includes a local cache and a distributed cache; The read strategy of the cache partition is: read data from the local cache first, and return to the distributed cache when there is a miss; the write strategy of the cache partition is: update strategy when writing.

10. A current limiting system based on a distributed cache strategy, characterized in that: Applied to a distributed cache system, the distributed cache system includes multiple cache partitions; the current limiting system based on the distributed cache strategy includes: A cache partition flow acquisition unit, used to acquire the real-time flow and historical flow of each cache partition; A traffic prediction unit, used for inputting the real-time traffic and the historical traffic into a traffic prediction model to obtain the predicted traffic at the next moment; A flow limiting unit is used to determine the flow limiting parameters of the cache partition based on the predicted flow, and to limit the flow at the next moment based on the flow limiting parameters.

Images