Current limiting method and system based on distributed cache strategy

Abstract

The present invention provides a flow limiting method and system based on a distributed cache strategy, which belongs to the field of distributed flow limiting technology and is applied to a distributed cache system, wherein the distributed cache system includes multiple cache partitions; the method includes: obtaining the real-time flow and historical flow of each cache partition; inputting the real-time flow and the historical flow into a flow prediction model to obtain the predicted flow at the next moment; determining the flow limiting parameters of the cache partition based on the predicted flow, and limiting the flow at the next moment based on the flow limiting parameters. The present invention uses machine learning technology to automatically adjust the flow limiting parameters, which can accurately control the data flow, prevent system overload, and avoid resource waste caused by excessive restrictions, significantly improving the flow management efficiency and effect of the distributed cache system.

Description

Technical Field

[0001] The present invention relates to the technical field of distributed current limiting, and in particular to a current limiting method and system based on a distributed caching strategy. Background Art

[0002] In today's rapidly evolving digital and networked world, data traffic management has become a major challenge for network service providers and enterprises. In areas such as cloud services, video streaming, online gaming, and e-commerce, effectively managing and controlling traffic to avoid network congestion and service degradation has become a key factor in maintaining competitiveness and user satisfaction.

[0003] The current limiting methods in existing technologies mainly focus on two categories: single-node current limiting and centralized current limiting. Single-node current limiting is usually implemented at the local server or application level, such as using a leaky bucket algorithm or a token bucket algorithm to control the request rate entering the system. Centralized current limiting strategies are managed on a central server or a dedicated current limiting service to uniformly schedule the request processing capacity of the entire network or system. There are still some shortcomings in distributed and highly dynamic network environments: (1) Lack of flexibility and scalability: Single-node and centralized current limiting technologies are difficult to flexibly cope with large-scale or geographically distributed network environments. These technologies have certain limitations in scenarios where current limiting strategies need to be dynamically adjusted to respond to real-time network status changes. (2) Single point of failure problem: In a centralized current limiting solution, all decision-making and data processing rely on the central node. Once the central node fails, the current limiting function of the entire system may fail directly. (3) Processing delay: When processing cross-regional traffic, the centralized system may not be able to respond to traffic changes in real time due to data transmission and processing delays, thereby affecting service quality and user experience.

[0004] Therefore, there is an urgent need to provide a current limiting method and system based on a distributed caching strategy to achieve accurate current limiting of large-scale and highly variable network traffic. Summary of the Invention

[0005] In view of this, it is necessary to provide a current limiting method and system based on a distributed caching strategy to solve the technical problem that single-node current limiting and centralized current limiting in the existing technology have poor current limiting effects when facing large-scale and highly variable network traffic.

[0006] On one hand, in order to solve the above technical problems, the present invention provides a current limiting method based on a distributed cache strategy, which is applied to a distributed cache system, wherein the distributed cache system includes multiple cache partitions; the method includes:

[0007] Obtaining real-time traffic and historical traffic of each cache partition;

[0008] Inputting the real-time traffic and the historical traffic into a traffic prediction model to obtain the predicted traffic at the next moment;

[0009] A flow limiting parameter of the cache partition is determined based on the predicted flow, and flow at a next moment is limited based on the flow limiting parameter.

[0010] In a possible implementation, determining the flow limiting parameter of the cache partition based on the predicted flow includes:

[0011] Configuring a traffic class handle for the cache partition based on a traffic controller, wherein the traffic class handle includes a preset flow rate configuration;

[0012] The preset flow rate configuration is adjusted based on the predicted flow to obtain the flow limiting parameter.

[0013] In a possible implementation, the method further includes:

[0014] The business importance and service level agreement of the real-time traffic are obtained, and the business importance and the service level agreement are input into a priority determination model to obtain traffic priority.

[0015] In a possible implementation, the method further includes:

[0016] Obtaining response time of pending requests, throughput of the distributed cache system, and user satisfaction;

[0017] Determining whether the traffic prediction model needs to be retrained based on the response time, the throughput, and the user satisfaction;

[0018] If so, the traffic prediction model is retrained based on the historical traffic.

[0019] In a possible implementation, before determining the flow limiting parameter of the cache partition based on the predicted flow, the method further includes:

[0020] Obtaining a request type and an expected access frequency of a request to be responded to, and determining partition parameters of the plurality of cache partitions based on the request type and the expected access frequency;

[0021] The partition parameters are optimized based on the real-time traffic and the system load of the distributed cache system to obtain optimized partition parameters.

[0022] In a possible implementation, optimizing the partition parameters based on the real-time traffic and the system load to obtain the optimized parameters includes:

[0023] Determining a required load of the cache partition based on the real-time traffic and the system load;

[0024] Obtaining a load threshold of the cache partition, and determining whether the cache partition needs to be adjusted based on the load threshold and the required load;

[0025] If so, the partition parameters are optimized based on a load balancing algorithm to obtain the optimized parameters.

[0026] In a possible implementation, the method further includes:

[0027] Obtaining a cache hit rate and a response time of the cache partition, and determining a first cache evaluation result based on the cache hit rate and the response time;

[0028] Obtaining user feedback and behavior analysis, and determining a second cache evaluation result based on the user feedback and the behavior analysis;

[0029] Based on the first cache evaluation result and the second cache evaluation result, it is determined whether partition parameters need to be adjusted; if so, the partition parameters are adjusted.

[0030] In a possible implementation, the distributed cache system further includes multiple data centers, and the correspondence between the data centers and the cache partitions is one-to-many. The method further includes:

[0031] Get the source IP address and user ID of the request to be responded to;

[0032] determining a geographic location of the request to be responded to based on the source IP address and the user ID;

[0033] A target data center among the plurality of data centers is determined based on the geographic location.

[0034] In a possible implementation, the cache partition includes a local cache and a distributed cache;

[0035] The read strategy of the cache partition is: read data from the local cache first, and return to the distributed cache when there is a miss; the write strategy of the cache partition is: update strategy when writing.

[0036] The present invention also provides a current limiting system based on a distributed cache strategy, which is applied to a distributed cache system, wherein the distributed cache system includes multiple cache partitions; the current limiting system based on a distributed cache strategy includes:

[0037] A cache partition traffic acquisition unit, configured to acquire the real-time traffic and historical traffic of each cache partition;

[0038] A traffic prediction unit, configured to input the real-time traffic and the historical traffic into a traffic prediction model to obtain a predicted traffic at the next moment;

[0039] A flow limiting unit is used to determine a flow limiting parameter of the cache partition based on the predicted flow, and to limit the flow at a next moment based on the flow limiting parameter.

[0040] The beneficial effects of the present invention are as follows: the flow limiting method based on the distributed cache strategy provided by the present invention obtains the real-time flow and historical flow of the cache partition, and determines the predicted flow at the next moment based on the real-time flow and historical flow, and predicts the flow of the rapidly changing network environment through the flow prediction model, thereby obtaining accurate predicted flow, and then determining the flow limiting parameters of the cache partition based on the predicted flow, thereby achieving accurate flow limiting of the flow at the next moment, which can significantly improve the flow management efficiency and effect of the distributed cache system, and provide more flexible and efficient support for modern network services. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 A schematic flow chart of an embodiment of a current limiting method based on a distributed cache strategy provided by the present invention;

[0043] Figure 2 For the present invention Figure 1 A schematic diagram of an embodiment of a flow chart for determining the current limiting parameters of the cache partition in step S103;

[0044] Figure 3 A schematic diagram of a flow chart of an embodiment of optimizing a traffic prediction model provided by the present invention;

[0045] Figure 4 A schematic diagram of a flow chart of an embodiment of setting and optimizing partition parameters of a cache partition provided by the present invention;

[0046] Figure 5 For the present invention Figure 4 A schematic flow chart of an embodiment of step S402;

[0047] Figure 6 A schematic diagram of a flow chart of an embodiment of further optimizing partition parameters provided by the present invention;

[0048] Figure 7A schematic diagram of a process flow for determining a target data center according to an embodiment of the present invention;

[0049] Figure 8 This is a schematic structural diagram of an embodiment of a current limiting system based on a distributed caching strategy provided by the present invention. DETAILED DESCRIPTION

[0050] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0051] It should be understood that the schematic drawings are not drawn to scale. The flowcharts used in the present invention illustrate operations implemented according to some embodiments of the present invention. It should be understood that the operations of the flowcharts can be implemented out of sequence, and steps that have no logical contextual relationship can be reversed in order or implemented simultaneously. In addition, those skilled in the art, guided by the content of the present invention, can add one or more other operations to the flowcharts or remove one or more operations from the flowcharts. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and / or processor systems and / or microcontroller systems.

[0052] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0053] The present invention provides a current limiting method and system based on a distributed caching strategy, which are described below.

[0054] Figure 1 A flow chart of an embodiment of a current limiting method based on a distributed cache strategy provided by the present invention is applied to a distributed cache system, which includes multiple cache partitions; Figure 1 As shown in the figure, the current limiting methods based on the distributed cache strategy include:

[0055] S101, obtaining the real-time traffic and historical traffic of each cache partition;

[0056] S102: Input the real-time traffic and historical traffic into the traffic prediction model to obtain the predicted traffic at the next moment;

[0057] S103: Determine a flow limiting parameter of the cache partition based on the predicted flow, and limit the flow at the next moment based on the flow limiting parameter.

[0058] The method for obtaining real-time traffic in step S101 is: deploying a dedicated monitoring agent in each cache partition, and accurately recording the arrival time, processing time, source and target information of each request to be responded to based on the monitoring agent, so as to obtain real-time traffic.

[0059] Specifically, the monitoring agent is deployed through Prometheus.

[0060] The historical traffic is obtained in step S102 by obtaining the historical traffic from a data warehouse that stores historical traffic data. The data warehouse may be Apache Hadoop, which supports subsequent large-scale data storage and complex queries.

[0061] Since the traffic prediction model is a time series prediction, in a preferred embodiment of the present invention, the model structure of the traffic prediction model is a Long Short-Term Memory (LSTM) network.

[0062] It should be understood that: before step S102, the constructed initial traffic prediction model needs to be trained, tested and verified to obtain a traffic prediction model for predicting traffic.

[0063] Among them, before training, testing and verifying the initial traffic prediction model, it is necessary to first build a data set, and then perform training, testing and verification based on the data set.

[0064] Specifically, a sliding window approach is used to create a dataset consisting of multiple input (feature) and output (label) pairs. Inputs include time, request volume, and response time.

[0065] Before using the dataset, it needs to be preprocessed, which includes but is not limited to data quality checking, data conversion, and standardization.

[0066] Data quality checks include missing value checking and outlier handling. Missing value checking ensures there are no missing data points in the time series. If missing, you can choose to fill in the missing data points (for example, using the previous value, mean, or median) or delete them. Outlier handling involves identifying and addressing outliers, which may be caused by measurement errors or other atypical events and may adversely affect model training.

[0067] Data transformation and standardization specifically include: 1. It is beneficial to standardize or normalize the data to a uniform scale. For example, you can use Min-Max scaling or Z-score normalization. 2. Encoding: If the data contains categorical features (such as day of the week, whether it is a holiday, etc.), these features need to be converted into a format that the model can process using one-hot encoding or label encoding.

[0068] It should be noted that the time span of historical traffic can be set according to the actual application scenario, such as 12 hours or 24 hours, and is not specifically limited here.

[0069] To improve the accuracy of traffic prediction, in some embodiments of the present invention, additional time features, such as weekdays, weekends, or statutory holidays, may be added to real-time and historical traffic to help the model better learn the impact of time variations. Specifically, real-time and historical traffic may also include traffic trend features.

[0070] In some embodiments of the present invention, the specific process of training, testing, and validating the constructed initial traffic prediction model is as follows:

[0071] Data segmentation: Ensure that the dataset is arranged in chronological order. For time series data, it is most important to maintain temporal continuity to avoid prediction bias caused by data temporal disorder.

[0072] Data partitioning: Based on the data volume, select 20%-30% of the data as the test set and the rest as the training set. For example, for data from 2023, select the first 9 months of data to train the model and the last 3 months of data to test it.

[0073] Model configuration: Configure the LSTM model structure.

[0074] By adjusting the following parameters:

[0075] Number of layers and neurons: In this example, 50 to 100 neurons are set, which can be increased or decreased based on the number of features and resource size.

[0076] Learning rate: Set to 0.001. If the performance is unstable during training, lower it appropriately.

[0077] Batch size: The number of samples in each batch is directly related to memory usage and training speed. The default setting is 32, which can be adjusted appropriately based on the actual GPU computing power. If the computing power is sufficient, 64 or 128 can be selected. Find the balance between speed and performance through specific experiments. When choosing, refer to the following specifications:

[0078] Optimizer: The Adam optimizer is selected. It has good algorithm optimization performance for simple prediction scenarios and supports automatic adjustment of the learning rate, which is suitable for the current prediction model LSTM.

[0079] The specific training process is as follows:

[0080] After the above configuration, follow the following process to load preprocessed data, build the compiled model, train, and save the pretrained weights.

[0081] Set Epochs to 50 and perform 50 rounds of training using the historical dataset. Adjust the number of Epochs between 50 and 100 based on validation results.

[0082] Loss function: When compiling the model, specify the mean squared error (MSE) function. The smaller the loss value, the closer it is to the truth, and the better the prediction performance.

[0083] Early stopping function: When the loss value does not decrease or increase after multiple epochs, training needs to be terminated early to avoid resource waste and prevent model overfitting.

[0084] Training corrections: If the loss value shows a poor downward trend or early stopping occurs, you can increase or decrease the number of LSTM layers, adjust the number of neurons in each layer, switch optimizers, reduce the learning rate, increase the batch_size, and increase the number of epochs. If the parameters do not work well, consider adding more easily identifiable feature data to the training set.

[0085] The specific process of model validation is as follows:

[0086] The trained model is evaluated using mean square error (MSE) and root mean square error (RMSE). The smaller the values ​​of the two, the better the evaluation performance.

[0087] The specific process of model testing is as follows:

[0088] 1. Compare the performance differences on the training set and the test set. If the error on the test set is much higher than that on the training set, there may be overfitting.

[0089] 2. Use cross-validation and other methods to further confirm the stability and reliability of the model.

[0090] 3. Visualize results: Plot a comparison chart of actual values ​​against predicted values ​​to visually demonstrate the model's forecast accuracy and trend recognition capabilities. Analyze the distribution of forecast errors to check for systematic biases.

[0091] It should be noted that the prediction interval of the predicted traffic can be set or adjusted according to parameters such as the access speed of the request to be responded to, for example, the traffic prediction model automatically runs every five minutes.

[0092] Compared with the existing technology, the flow limiting method based on distributed caching strategy provided by the embodiment of the present invention obtains the real-time flow and historical flow of the cache partition, and determines the predicted flow at the next moment based on the real-time flow and historical flow. The flow of the rapidly changing network environment is predicted through the flow prediction model, and accurate predicted flow is obtained. The flow limiting parameters of the cache partition can be determined based on the predicted flow, and accurate flow limiting of the flow at the next moment can be achieved. It can significantly improve the flow management efficiency and effect of the distributed caching system, and provide more flexible and efficient support for modern network services.

[0093] In some embodiments of the present invention, Figure 2 As shown, determining the flow limiting parameters of the cache partition based on the predicted flow in step S103 includes:

[0094] S201, configuring a traffic class handle of a cache partition based on a traffic controller (Traffic Control, TC), where the traffic class handle includes a preset flow rate configuration;

[0095] S202: Adjust the preset flow rate configuration based on the predicted flow rate to obtain flow limiting parameters.

[0096] The traffic class handle of the cache partition configured based on the TC in step S201 is specifically:

[0097] Define queue rules and create queues based on them;

[0098] Define class rules, which include multiple root nodes, eth0 devices, and their corresponding traffic class handles, and preset traffic configuration for each traffic class handle;

[0099] Define application filters and add filter rules. The filter mounts the traffic class handle. The filter rules define the correspondence between the traffic class handle and each port.

[0100] The pseudo code of the above process is as follows:

[0101] “# Define queue discipline

[0102] # Set up a bandwidth allocation system based on htb (hierarchical token bucket)

[0103] # qdisc add Create a queue

[0104] # dev eth0 specifies the network interface eth0

[0105] # root hanlde 1: Set the queue discipline and handle ID to 1

[0106] # default 10 specifies the default class for unclassified traffic

[0107] tc qdisc add dev eth0 root handle 1: htb default 10

[0108] # Define class rules

[0109] # Create a traffic class handle of 1:1 under the root node (parent node 1) and specify a bandwidth of 50Mbps

[0110] tc class add dev eth0 parent 1: classid 1:1 htb rate 50mbit

[0111] # Create a traffic class handle of 1:2 under the root node (parent node 1) and specify a bandwidth of 20 Mbps.

[0112] tc class add dev eth0 parent 1: classid 1:2 htb rate 20mbit

[0113] # Change the bandwidth limit of class 1:1 on eth0 from 50Mbps to 100Mbps

[0114] tc class change dev eth0 parent 1: classid 1:1 htb rate 100mbit

[0115] # Define application filters

[0116] # Add filter rules

[0117] # filter add new filter

[0118] # dev eth0 distributes traffic on network interface eth0

[0119] # protocol ip specifies the protocol as ip. You can also use ipv6 or other network layer protocols.

[0120] # parent 1:0 refers to the filter mount point handle, associated with the root qdisc

[0121] # prio 1 Filter priority is 1, arranged as needed

[0122] # u32 Common and flexible filter matching mechanism

[0123] # match ip dst 192.168.1.100 / 32 matches the destination IP address 192.168.1.100. The matching rule can also be selected

[0124] src (source address), dport (destination port), port (source port), icmp type (ICMP message type), etc. The mask used (such as 0xffff) can be changed to support partial matching.

[0125] # flowid 1:1 directs traffic to the class with handle 1:1

[0126] tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ip dst192.168.1.100 / 32 flowid 1:1

[0127] # Direct all traffic destined for port 80 to the class with handle 1:2 in the qdisc queue.

[0128] tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ipdport 80 0xffff flowid 1:2”.

[0129] In some embodiments of the present invention, the current limiting parameters include a Quality of Service (QoS) level and a request queuing strategy.

[0130] Among them, QoS describes the service quality of the network, including various technical indicators such as delay, packet loss rate, throughput, etc., and is often used in network traffic management and priority setting.

[0131] In a specific embodiment of the present invention, step S202 is specifically as follows:

[0132] Obtain the high and low threshold traffic indicators. If the predicted traffic volume exceeds the high threshold, the default traffic rate configuration is switched to the corresponding low rate. For example, if the traffic volume exceeds 1000 but is less than 2000, the default traffic rate is 60 Mbps. If the predicted traffic volume is 1200, the default traffic rate is switched to 60 Mbps. If the predicted traffic volume is less than the low threshold, the default traffic rate is switched to the corresponding low-speed scenario.

[0133] It should be noted that steps S201 - S202 may be automated using a cron job.

[0134] Since the traffic of financial transactions and other businesses is of high importance, priority processing of such business traffic is required to ensure the normal operation of such businesses. Therefore, in some embodiments of the present invention, the current limiting method based on the distributed cache strategy further includes:

[0135] The business importance and service level agreement of real-time traffic are obtained and input into the prioritization model to obtain traffic priority.

[0136] The embodiment of the present invention can ensure priority response to important business traffic by performing differentiated processing on traffic and prioritizing real-time traffic.

[0137] It should be noted that the process of determining the flow limiting parameters in step S103 can also take traffic priority into consideration, that is, the flow limiting parameters of the cache partition are determined based on the predicted traffic and traffic priority to further improve the accuracy of the flow limiting parameters and their adaptability to business scenarios.

[0138] Since the priorities of different services may change over time, in some embodiments of the present invention, traffic priority can be dynamically adjusted based on real-time service needs and user behavior to further improve the real-time nature of traffic priority.

[0139] Specifically, the service priority of paying users can be increased when resources are scarce.

[0140] To ensure the accuracy of the predicted flow, in some embodiments of the present invention, Figure 3 As shown, the current limiting method based on the distributed cache strategy also includes:

[0141] S301, obtaining the response time of the pending request, the throughput of the distributed cache system, and the user satisfaction;

[0142] S302: Determine whether the traffic prediction model needs to be retrained based on response time, throughput, and user satisfaction;

[0143] S303: If yes, retrain the traffic prediction model based on historical traffic.

[0144] The retraining of the traffic prediction model in step S303 includes but is not limited to: adjusting model parameters and replacing model algorithms.

[0145] The embodiment of the present invention can adjust and update the traffic prediction model to adapt to changes in the network environment and new business needs by judging whether the traffic prediction model needs to be retrained based on response time, throughput and user satisfaction. As a result, the flow limiting parameters determined based on the traffic prediction model meet the response time, throughput and user satisfaction requirements during use, thereby improving the rationality and accuracy of the flow limiting parameters. By retraining the traffic prediction model, the embodiment of the present invention can maximize the processing capacity and resource utilization of the distributed cache system while maintaining the quality of network service, and has important adaptability and value for dynamic and changing network environments.

[0146] Since the predicted traffic needs to be written into the cache partition or read from the cache partition, in order to make the response of reading and writing the predicted traffic faster, in some embodiments of the present invention, such as Figure 4 As shown, before step S103, the following steps are also included:

[0147] S401: Obtain a request type and an expected access frequency of a request to be responded to, and determine partition parameters of multiple cache partitions based on the request type and the expected access frequency;

[0148] S402: Optimize partition parameters based on real-time traffic and the system load of the distributed cache system to obtain optimized partition parameters.

[0149] The request type can be static or dynamic. Dynamic requests are requests with frequent data updates, while static requests are requests with infrequent data updates. Static requests are more suitable for long-term caching.

[0150] The expected access frequency can be divided into high-frequency access and low-frequency access based on the frequency threshold. High-frequency access, such as popular video clips, needs to be retained in the cache for a longer time, while low-frequency access, such as API responses, has a shorter life cycle in the cache.

[0151] Partition parameters include but are not limited to cache size and data expiration policy.

[0152] The embodiment of the present invention determines the partition parameters of multiple cache partitions based on the obtained request type and expected access frequency, allowing the distributed cache system to adjust resource allocation according to the request type and expected access rate, thereby optimizing the response rate of pending requests and improving the efficiency of data writing and reading in the cache partitions.

[0153] Furthermore, the embodiment of the present invention is configured to optimize partition parameters based on real-time traffic and system load, and can dynamically expand or shrink partition parameters to ensure that the distributed cache system can continue to operate stably under load fluctuations during peak traffic periods, thereby ensuring the optimization of system performance and resource utilization.

[0154] In some embodiments of the present invention, Figure 5 As shown, step S402 includes:

[0155] S501, determining the required load of the cache partition based on real-time traffic and system load;

[0156] S502: Obtain a load threshold of a cache partition, and determine whether the cache partition needs to be adjusted based on the load threshold and the required load;

[0157] S503: If yes, optimize the partition parameters based on the load balancing algorithm to obtain optimized parameters.

[0158] The demand load in step S501 is the difference between the real-time traffic and the system load.

[0159] Among them, the load threshold in step S502 includes a minimum load and a maximum load, and the cache partition is adjusted specifically as follows: when the required load is less than the minimum load, the cache partition is shrunk; when the required load is greater than the maximum load, the cache partition is expanded.

[0160] The contraction process specifically includes: reducing the cache size of the cache partition or triggering data migration; the expansion process specifically includes: increasing the cache size of the cache partition or triggering data migration.

[0161] By determining the required load and dynamically determining whether partition parameters need adjustment based on the required load, the embodiments of the present invention can improve resource utilization. Furthermore, when optimizing partition parameters, a load balancing algorithm can be used to redistribute cache across nodes, ensuring a balanced load across nodes and further optimizing resource utilization efficiency.

[0162] In a specific embodiment of the present invention, in order to ensure the consistency of cache partition data, a consistent hashing algorithm is applied to process the data distribution between cache partition nodes, ensuring that the need for data redistribution is reduced when adding and removing cache nodes, thereby maintaining the stability of the cache partition.

[0163] Specifically, a hash ring technique is used to distribute data within a Redis cluster, ensuring even data distribution and optimizing load balancing. The consistent hashing algorithm minimizes data migration when adding or removing nodes, affecting only a small portion of the data on the hash ring, similar to the implementation in the Apache Cassandra database.

[0164] Furthermore, to avoid technical issues such as data loss caused by failures during cache partition allocation, critical data is replicated in multiple physical locations to ensure rapid service restoration in the event of any cache partition failure.

[0165] Specifically, Write-Ahead Logging (WAL) is a mechanism that writes data into a log before it is modified. This mechanism uses a WAL mechanism such as the one used by the PostgreSQL database to ensure that any database transaction is recorded before it is committed, so that these transactions can be recovered after a system crash.

[0166] Furthermore, when a distributed cache system fails, an automatic fault recovery mechanism can be configured. Specifically, for transient failures, such as network delays or temporary service unresponsiveness, the system will automatically retry requests. This is achieved by configuring a retry policy in the Nginx reverse proxy server or a service mesh such as Istio. For persistent failures, the system automatically switches to a hot standby server or service, such as using AWS Route 53 for DNS failover, to ensure the continuity of critical services.

[0167] Since the network environment is changing dynamically and the service requirements are also changing dynamically, in order to achieve the adaptability of the cache partition to the dynamically changing network environment and service requirements, in some embodiments of the present invention, such as Figure 6 As shown, the current limiting method based on the distributed cache strategy also includes:

[0168] S601: Obtain a cache hit rate and a response time of a cache partition, and determine a first cache evaluation result based on the cache hit rate and the response time;

[0169] S602: Obtain user feedback and behavior analysis, and determine a second cache evaluation result based on the user feedback and behavior analysis;

[0170] S603: Determine whether partition parameters need to be adjusted based on the first cache evaluation result and the second cache evaluation result; if so, adjust the partition parameters.

[0171] The embodiment of the present invention determines whether the partition parameters need to be adjusted based on the first cache evaluation result and the second cache evaluation result, and can judge whether the partition parameters meet the dynamically changing network parameters and business requirements. If they do not meet the requirements, the partition parameters are adjusted to ensure that the partition parameters adapt to changes in business requirements and changes in the network environment.

[0172] In a specific embodiment of the present invention, the distributed cache system further includes multiple data centers, and the correspondence between data centers and cache partitions is one-to-many, that is, multiple cache partitions and a data center are located in the same geographical location range.

[0173] Based on the above distributed cache system, in order to further improve the response speed of the cache partition to the request to be responded to, in some embodiments of the present invention, such as Figure 7 As shown, the current limiting method based on the distributed cache strategy also includes:

[0174] S701. Obtain the source IP address and user ID of the request to be responded to;

[0175] S702, determining the geographic location of the request to be responded to based on the source IP address and user ID;

[0176] S703: Determine a target data center among multiple data centers based on the geographical location.

[0177] The embodiment of the present invention first determines a target data center based on the geographical location of the request to be responded to, and then optimizes partition parameters of multiple cache partitions corresponding to the target data center.

[0178] The embodiment of the present invention determines the target data center based on the geographical location, thereby improving the level of refined management of traffic and effectively improving the efficiency of traffic management in the distributed cache system.

[0179] It should be noted that: to ensure the failure recovery rate, key data can be asynchronously replicated between different data centers, using mechanisms such as MySQL replication to implement master-slave replication to ensure data consistency and continuous availability.

[0180] To further improve the response rate of pending requests, in some embodiments of the present invention, the cache partition includes a local cache and a distributed cache;

[0181] The read policy of the cache partition is to read data from the local cache first and return it to the distributed cache if it misses. The write policy of the cache partition is to update data on write (write-through) policy.

[0182] The embodiment of the present invention reduces data access delay and improves data retrieval speed based on local cache by setting cache partitions including two-level local cache and distributed cache, and improves data availability and fault tolerance based on distributed cache.

[0183] The local cache uses memory caching technology, such as Redis or Memcached. Its configuration includes setting an appropriate expiration policy, such as an automatic elimination mechanism based on the LRU algorithm.

[0184] Distributed cache is deployed in multiple geographical locations and uses consistent hashing technology for load balancing. This allows for multiple copies of data to be stored, improving recovery speed.

[0185] It should be noted that local and distributed caches must be cleaned regularly to regularly check and purge expired or infrequently accessed data. At the same time, cache data at all levels must be synchronized to ensure consistency and the latest status of the data.

[0186] To ensure efficient operation and responsiveness of the multi-level cache, a performance monitoring and optimization mechanism is added. The detailed implementation steps are as follows:

[0187] 1) Monitoring system deployment

[0188] Monitoring tool selection: Choose appropriate monitoring tools, such as Prometheus combined with Grafana to visualize monitoring data, or use commercial solutions such as New Relic and Datadog to provide comprehensive performance monitoring.

[0189] Performance indicator definition:

[0190] Hit rate: A key indicator for measuring caching effectiveness. A high hit rate indicates that most requests are cached successfully, reducing the pressure on the backend database.

[0191] Latency: Monitors the response time of each cache request and the average time it takes for data to be read from the cache. This is key to responding to user requests quickly.

[0192] Capacity Utilization: Track cache resource usage, including memory and disk space usage, to ensure the cache tier has sufficient resources to handle the expected load.

[0193] Monitoring frequency and alarm thresholds: Set the frequency of data collection and the alarm threshold when performance is lower than expected. When performance data is abnormal, the alarm can be triggered in time and the administrator can be notified.

[0194] 2) Performance Tuning

[0195] Dynamic resource allocation: The size and quantity of caches are dynamically adjusted based on real-time monitoring data. For example, if a cache node frequently reaches its capacity limit, the system can automatically expand the cache capacity or add more nodes.

[0196] Configuration optimization: Optimize cache configuration parameters, such as adjusting the cache expiration policy or updating the data refresh frequency, to adapt to changing usage patterns.

[0197] Automated optimization scripts: Develop automated scripts or use intelligent tuning tools, such as AWS Auto Scaling, to automatically adjust resource allocation based on load changes to ensure the optimal balance between performance and cost.

[0198] 3) Feedback mechanism

[0199] User feedback collection: Collect user feedback on cache performance through user surveys, online feedback tools, or customer support services, especially after cache updates or configuration changes.

[0200] System log analysis: System logs provide detailed records of cache operations. Analyzing these logs can help identify cache-level problems, such as frequent cache penetration or cache avalanche.

[0201] Continuous improvement process: Establish a mechanism for regular review and iterative improvement, incorporate user feedback and log analysis results into the continuous optimization of caching strategies and architecture, and ensure the continuous adaptability and performance improvement of the system.

[0202] In order to further improve the fault tolerance of the distributed cache system, in some embodiments of the present invention,

[0203] 1) System redundancy design: Multiple instances of key components can be deployed, for example, using cluster configurations. For example: 1) Use a Kubernetes cluster to manage multiple copies of microservices to ensure that the service can continue to run even if a single component fails.

[0204] 2) Fault Isolation Strategy: Use a microservices architecture to separate different service units, ensuring that failure of one component of the system does not affect the normal operation of other unrelated components. For example, using Docker containerization technology to deploy services, each service runs in an independent container, isolating the spread of failures.

[0205] In summary, the distributed caching strategy-based flow control method provided by the embodiments of the present invention intelligently classifies data and allocates resources based on traffic attributes and sources, ensuring that all types of requests receive optimal processing. It also uses machine learning techniques to automatically adjust flow control parameters. This adaptive mechanism precisely controls data flow, preventing system overload and avoiding resource waste caused by excessive restrictions. Furthermore, the multi-level caching strategy and advanced fault tolerance mechanisms not only enhance system scalability and responsiveness, but also improve system reliability and resilience.

[0206] The embodiments of this invention innovatively provide a flexible, reliable, and efficient method for handling large-scale and highly variable network traffic. This method significantly improves the efficiency and effectiveness of traffic management in distributed systems, providing more stable, flexible, and efficient support for modern network services. This approach is particularly well-suited to the needs of modern distributed systems and cloud infrastructure. Furthermore, the implementation of this method helps reduce overall system maintenance costs and operational complexity, providing network service providers and enterprises with a powerful tool for ensuring the continuity and high availability of critical services.

[0207] In order to better implement the current limiting method based on the distributed cache strategy in the embodiment of the present invention, on the basis of the current limiting method based on the distributed cache strategy, the embodiment of the present invention also provides a current limiting system based on the distributed cache strategy, which is applied to the distributed cache system. The distributed cache system includes multiple cache partitions; Figure 8 As shown, the current limiting system 800 based on the distributed cache strategy includes:

[0208] The cache partition traffic acquisition unit 801 is used to acquire the real-time traffic and historical traffic of each cache partition;

[0209] The traffic prediction unit 802 is used to input the real-time traffic and historical traffic into the traffic prediction model to obtain the predicted traffic at the next moment;

[0210] The flow limiting unit 803 is configured to determine a flow limiting parameter of the cache partition based on the predicted flow, and to limit the flow at the next moment based on the flow limiting parameter.

[0211] The current limiting system 800 based on the distributed cache strategy provided in the above embodiment can implement the technical solution described in the above embodiment of the current limiting method based on the distributed cache strategy. The specific implementation principles of the above modules or units can be found in the corresponding contents in the above embodiment of the current limiting method based on the distributed cache strategy, which will not be repeated here.

[0212] Those skilled in the art will appreciate that all or part of the process flow of the above-described method embodiment can be implemented by instructing related hardware (such as a processor, controller, etc.) through a computer program, and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a magnetic disk, an optical disk, a read-only memory, or a random access memory.

[0213] The above is a detailed introduction to a current limiting method and system based on a distributed caching strategy provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A current limiting method based on a distributed cache strategy, characterized in that: Applied to a distributed cache system, the distributed cache system includes multiple cache partitions; the method includes: Obtaining real-time traffic and historical traffic of each cache partition; Inputting the real-time traffic and the historical traffic into a traffic prediction model to obtain the predicted traffic at the next moment; Obtaining the business importance and service level agreement of the real-time traffic, and inputting the business importance and the service level agreement into a priority determination model to obtain traffic priority; The flow limiting parameters of the cache partition are determined based on the predicted flow and flow priority, and the flow at the next moment is limited based on the flow limiting parameters. The flow limiting parameters include flow rate index, service quality level and request queuing strategy.

2. The current limiting method based on the distributed cache strategy according to claim 1 is characterized in that: The method further comprises: Obtaining response time of pending requests, throughput of the distributed cache system, and user satisfaction; Determining whether the traffic prediction model needs to be retrained based on the response time, the throughput, and the user satisfaction; If so, the traffic prediction model is retrained based on the historical traffic.

3. The current limiting method based on the distributed cache strategy according to claim 1 is characterized in that: Before determining the flow limiting parameters of the cache partition based on the predicted flow and the flow priority, the method further includes: Obtaining a request type and an expected access frequency of a request to be responded to, and determining partition parameters of the plurality of cache partitions based on the request type and the expected access frequency; The partition parameters are optimized based on the real-time traffic and the system load of the distributed cache system to obtain optimized partition parameters.

4. The current limiting method based on the distributed cache strategy according to claim 3 is characterized in that: Optimizing the partition parameters based on the real-time traffic and the system load of the distributed cache system to obtain optimized partition parameters includes: Determining a required load of the cache partition based on the real-time traffic and the system load; Obtaining a load threshold of the cache partition, and determining whether the cache partition needs to be adjusted based on the load threshold and the required load; If so, the partition parameters are optimized based on a load balancing algorithm to obtain the optimized partition parameters.

5. The current limiting method based on the distributed cache strategy according to claim 3 is characterized in that: The method further comprises: Obtaining a cache hit rate and a response time of the cache partition, and determining a first cache evaluation result based on the cache hit rate and the response time; Obtaining user feedback and behavior analysis, and determining a second cache evaluation result based on the user feedback and the behavior analysis; Based on the first cache evaluation result and the second cache evaluation result, it is determined whether partition parameters need to be adjusted; if so, the partition parameters are adjusted.

6. The current limiting method based on a distributed cache strategy according to any one of claims 1 to 5, characterized in that: The distributed cache system further includes a plurality of data centers, and the corresponding relationship between the data centers and the cache partitions is one-to-many. The method further includes: Get the source IP address and user ID of the request to be responded to; determining a geographic location of the request to be responded to based on the source IP address and the user ID; A target data center among the plurality of data centers is determined based on the geographic location.

7. The current limiting method based on the distributed cache strategy according to claim 1 is characterized in that: The cache partition includes a local cache and a distributed cache; The read strategy of the cache partition is: read data from the local cache first, and return to the distributed cache when there is a miss; the write strategy of the cache partition is: update strategy when writing.

8. A current limiting system based on a distributed cache strategy, characterized in that: Applied to a distributed cache system, the distributed cache system includes multiple cache partitions; the current limiting system based on the distributed cache strategy includes: A cache partition traffic acquisition unit, configured to acquire the real-time traffic and historical traffic of each cache partition; A traffic prediction unit is configured to input the real-time traffic and the historical traffic into a traffic prediction model to obtain the predicted traffic at the next moment; obtain the business importance and service level agreement of the real-time traffic, and input the business importance and the service level agreement into a priority determination model to obtain traffic priority; A flow limiting unit is used to determine the flow limiting parameters of the cache partition based on the predicted flow and flow priority, and to limit the flow at the next moment based on the flow limiting parameters, wherein the flow limiting parameters include flow rate index, service quality level and request queuing strategy.

Images