Low-overhead and accurate learning-based Web cache eviction strategy

Abstract

The invention provides a low-overhead and accurate learning-based Web cache expelling strategy, and belongs to the technical field of machine learning. The Web cache can effectively reduce request delay and network traffic, and an eviction strategy is the core of the Web cache. Most of the prior art aims to reduce the loss rate. However, the use of machine learning greatly increases the computation overhead of a cache policy, which seriously restricts deployment in a production environment. In order to balance the loss rate and the calculation overhead, the invention can realize the lowest object loss rate and byte loss rate in almost all data sets while realizing low calculation overhead. The method comprises the following steps: firstly, the 3L-Cache dynamically adjusts the training frequency under the condition that the precision is not influenced by reducing the computing resource waste caused by unnecessary training updating; then, an eviction method is provided. According to the method, through a bidirectional sampling strategy, an expelling prediction ratio generator and low-overhead management, prediction overhead is remarkably reduced, and meanwhile accurate object expelling is achieved.

Description

Technical Field

[0001] The present invention is based on machine learning technology and studies a learning-based eviction strategy with low computational overhead and low loss rate. First, the present invention maintains a sliding window, which records the historical information of the object, which will be used for model training and prediction; then, through a two-way sampling strategy, the candidate objects for eviction are sampled, and the trained prediction model (gradient boosting machine) is used to predict the next arrival time of the object; finally, the eviction count is calculated based on the number of candidate objects for eviction and the eviction prediction ratio, the number of objects that can be evicted is determined, and the farthest arriving object is evicted based on the prediction result until the cache space is sufficient to store the new object. After the eviction operation is completed, the prediction results of the objects that have not been evicted are updated and managed for the next eviction. The present invention belongs to the field of cache systems, and specifically relates to technologies such as machine learning and eviction algorithms. Background Art

[0002] Web caches play a critical role in modern production environments, significantly reducing network traffic and speeding up response times for client requests. For example, in 2020, Netflix’s demand accounted for 1% of the total transatlantic cable bandwidth, and 95% of its traffic came directly from caches. At the heart of any cache system is its eviction policy, which determines which data needs to be cached and which data needs to be removed to make room. The performance of this policy is typically measured using two key metrics: (1) object loss rate, which represents the proportion of requests that result in cache misses; and (2) byte loss rate, which reflects the amount of cache miss data. A lower object loss rate means faster request responses, while a lower byte loss rate means fewer data requests to the origin server, reducing the burden on the backbone network.

[0003] Over the years, from heuristic eviction strategies to current learning-based strategies, cache miss rates have been strongly reduced, with average byte miss rates and object miss rates reduced by 7.2% and 13.3% respectively compared to LRU. However, the average computational overhead can be 172.5 times higher than LRU. The additional computational overhead brought by machine learning limits the deployment of learning-based strategies in production environments. Therefore, this work seeks opportunities to reduce the computational overhead to the same order of magnitude as the heuristic strategy while retaining the advantages of the learning-based strategy in terms of miss rate. It is worth noting that reducing the computational overhead also means an increase in the efficiency of processing requests, which can double the throughput of the cache system.

[0004] We classify learning-based strategies into three categories. 1) Policy-level learning (e.g., LeCaR, CACHEUS) uses reinforcement learning to directly learn eviction decisions. It has difficulty achieving optimal miss rates due to the delayed perception of workload changes generated by reinforcement learning. 2) Group-level learning (e.g., GL-Cache) clusters similar objects into groups and performs learning and eviction at the group level to greatly reduce computational overhead. However, due to the limitations of the grouping method and the definition of group utility, this strategy only performs well on limited datasets. 3) Object-level learning (e.g., LRB, HALP, Raven, and Parrot) performs training and prediction for each object. By predicting the reuse distance, this type of strategy can mimic Belady's MIN algorithm, which makes the "optimal decision" by evicting the object with the farthest future request among all objects in the cache. Although the object-level learning category has the potential to achieve the lowest miss rate among all learning strategies, its most fine-grained learning incurs the highest computational overhead. We find that the computational overhead for training and prediction accounts for more than 70%, and their proportions vary with the cache size.

[0005] Considering the problem that the current cache strategy is difficult to achieve a good balance between computational overhead and loss rate, the present invention proposes a learning-based object-level eviction strategy with low computational overhead and low loss rate. First, the present invention maintains a sliding window and controls the sampling frequency of training samples in the sliding window to control the training frequency, thereby significantly reducing the computational overhead brought by model training; then, through a bidirectional sampling strategy, the eviction candidate objects are sampled, and the trained prediction model (gradient boosting machine) is used to predict the next arrival time of the object. By increasing the number of object predictions each time, the parallelism of the prediction is improved to reduce the computational overhead of each object; finally, the eviction count is calculated based on the number of eviction candidate objects and the eviction prediction ratio, the number of objects that can be evicted is determined, and the farthest arriving object is evicted based on the prediction results until the cache space is sufficient to store the new object. After the eviction operation is completed, the prediction results of the non-eviction objects are updated and managed for the next eviction. By setting a large eviction prediction ratio to reduce the prediction overhead, and by reusing the prediction results, the optimal loss rate is guaranteed. Summary of the invention

[0006] The present invention is based on machine learning technology and studies a learning-based eviction strategy with low computational overhead and low loss rate. First, the present invention maintains a sliding window, which records the historical information of the object, and this information will be used for model training and prediction; then, a two-way sampling strategy is used to sample eviction candidate objects, and the trained prediction model (gradient boosting machine) is used to predict the next arrival time of the object; finally, the eviction count is calculated based on the number of eviction candidate objects and the eviction prediction ratio, the number of objects that can be evicted is determined, and the farthest arriving objects are evicted based on the prediction results until the cache space is sufficient to store the new object. After the eviction operation is completed, the prediction results of the non-eviction objects are updated and managed for the next eviction. This method mainly uses machine learning technology to simulate the theoretically optimal cache strategy Belady, and achieves performance that is better than all existing cache strategies. As shown in the attached Figure 1 As shown, the method can be mainly divided into four components: training data collection, prediction model, sampling of eviction candidates and efficient management of eviction candidates.

[0007] (1) Collection of training data

[0008] The prediction model of the present invention uses online training, and the prediction model needs to be updated in real time to ensure the reliability of the prediction. In order to ensure the update of the model, it is necessary to set the sliding window size to record historical request information, and the training data will be sampled from the sliding window. If the window is too small, the amount of information will be insufficient, and it will be difficult to train a good model; if the window is too large, some of the training data will be too old, and the trained model will accurately predict the access mode of the current object. Therefore, a suitable window size is necessary.

[0009] (2) Prediction Model

[0010] The prediction model is mainly used to predict the next arrival time of an object, providing a basis for the expulsion of the object. However, considering that a complex model will generate a large amount of computational overhead, the present invention selects a gradient booster as the prediction model, which can achieve efficient training and prediction on the CPU, meeting the present invention's requirement for low computational overhead.

[0011] (3) Eviction of cached objects

[0012] First, the present invention proposes a new sampling strategy for sampling and eviction candidate objects. It is mainly divided into two steps: one is directional sampling, which will sample based on the waiting time of the object; the other is fast sampling. Considering that the request for the object is zipf distributed, the missed objects may mostly be unpopular objects. Therefore, it is necessary to sample new objects in time and use the unpopular objects for eviction to prevent them from staying in the cache space for too long, which leads to an increase in the loss rate. Prioritizing the sampling of unpopular objects can avoid the mistaken eviction of popular objects, which can ensure the stability of the cache algorithm and achieve a low loss rate. Then, the next arrival time of the predicted sampled objects is sorted according to the prediction results. Finally, the farthest arriving object is evicted.

[0013] (4) Management of candidates for expulsion

[0014] The present invention will record the prediction results of the objects that have not been evicted for subsequent eviction, which will result in a large number of candidate objects for eviction. The traversal method cannot efficiently evict the farthest reaching object among these objects, so the present invention needs to manage it. The management method is mainly through heaps and hash tables. The maximum heap can provide efficient query and can quickly find the farthest reaching object from a large number of candidate objects for eviction. The hash table records the prediction results of the object to ensure the accuracy of the data.

[0015] Compared with the prior art, the present invention has the following obvious advantages and beneficial effects:

[0016] First, compared with the heuristic cache strategy, the present invention uses machine learning to simulate the theoretical optimal algorithm Belady. Therefore, its loss rate is much lower than that of the heuristic algorithm.

[0017] Secondly, the heuristic cache strategy usually performs well only under specific workloads, but it is difficult to adapt to the workload of the load, which makes it unstable. The present invention only predicts based on the access pattern of the object, and its performance under different workloads is more stable.

[0018] Finally, compared with the existing object-level learning-based caching strategy, the present invention can not only achieve a lower loss rate, but also significantly reduce the computational overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Detailed architecture diagram of the cache system

[0020] Figure 2 Sampling strategy flow chart DETAILED DESCRIPTION

[0021] According to the above description, the following is a specific implementation process, but the scope of protection of this patent is not limited to this implementation process.

[0022] Step 1: Collection of training data

[0023] Step 1: Collection of training data

[0024] Step 1.1: Sliding Window Size

[0025] A method for dynamically adjusting the sliding window size based on a hyperparameter h is proposed, where h is the ratio of objects in the sliding window to objects in the cache. The default value of h is 5.

[0026] Step 1.2: Sample and label training data

[0027] For each incoming request, an object is randomly sampled and the sampling time is recorded. The sampling rules are as follows: (1) In order to enrich the source of training data, random sampling and recording are performed within the sliding window; (2) When sampling an object with a record, the current information is not recorded again.

[0028] When the marked object is hit, a training sample is obtained and its record is deleted. When the object has not been requested and leaves the sliding window, a training sample is obtained, and its training label is its waiting time in the sliding window. After accumulating 65536 training samples, the model is updated.

[0029] Step 2: Prediction Model

[0030] The present invention uses a machine learning model gradient booster that is efficient on the CPU.

[0031] Input features: The model input uses 34 input features, including 32 historical arrival time intervals, age, and object size.

[0032] Prediction target: The prediction is the next arrival time interval of the object, and the predicted label is taken as logarithm. The next arrival time of the object is determined by the following formula:

[0033]

[0034] where t next Indicates the predicted next arrival time of the object. represents the predicted value of the model, t cur represents the current time, and e is Napier's constant, which is approximately 2.718.

[0035] Loss function: The present invention uses mean square error as the loss function, and the formula is as follows:

[0036]

[0037] where y i represents the label of the i-th training sample, Represents the predicted value of the i-th sample.

[0038] Step 3: Eviction of cached objects

[0039] Objects that are evicted by the offline optimal algorithm Belady are defined as unpopular objects. Therefore, in order to improve the performance of the cache algorithm, it is necessary to increase the sampling ratio of unpopular objects.

[0040] Step 3.1: Sampling of eviction candidates

[0041] Bidirectional sampling strategy. It includes: (1) Directed reset sampling, sampling objects with the longest waiting time and the lowest number of accesses. When analyzing the data set using the Belady algorithm, it was found that objects with the longest waiting time and the lowest number of accesses were more likely to be evicted. (2) Fast sampling, sampling newly arrived objects. Based on the analysis of the public data set Twitter, the one-time hit miracle can be as high as 95% of the newly arrived objects. The one-time hit miracle object means that the object will not be accessed again after being accessed once. The Belady algorithm will give priority to evicting such objects, and caching such objects will only waste cache space. However, the queue is maintained based on LRU, and the new object is at the end of the queue. Directed reset sampling needs to scan the entire queue to obtain the new object. At this time, the algorithm cannot efficiently evict such objects, so it needs to be sampled separately. Therefore, the proportion of unpopular objects in the newly arrived objects is high, which meets the algorithm's standard for sampling unpopular objects.

[0042] Step 3.1.1: Directed reset sampling

[0043] At the beginning of each round of sampling, the cache queue is divided into 10 groups according to its length. Therefore, sampling starts from the first group at the head of the queue, the sampling step is initially set to 1, and 256 objects are sampled each time. As the sampling pointer moves to the end of the queue, the frequency of object access increases. Therefore, compared with the previous group, the sampling step in each subsequent group is doubled, and the value sampling history request count is less than r max In addition, the number of requests encountered during the scan does not exceed r min The object will be sampled immediately. After completing 10 groups of sampling, the sampling pointer will be reset back to the head of the queue.

[0044] Based on the cache queue object request frequency distribution, r min Set to the number of requests at its 10th percentile to sample objects with fewer accesses in groups 2-10; r max Set to the 50th percentile of the number of requests to filter out objects with high request counts in the 2-10 group.

[0045] Step 3.1.2: Quick sampling

[0046] An additional queue is used to record the IDs of newly arrived objects (called the record queue). Fast sampling samples newly arrived objects based on first-in, first-out. After an object is sampled, it is removed from the record queue until the space occupied by objects in the record queue is less than 5% or the number of sampled objects exceeds 512. Once either of these two conditions is met, fast sampling ends.

[0047] Step 3.2: Generation of eviction prediction ratio

[0048] The present invention proposes to record the prediction results of the candidate objects that have not been evicted after each eviction for subsequent eviction. In addition, the present invention also needs to set the eviction prediction ratio, which will change dynamically with each sampling. When there are no eviction candidates, the initial value of the eviction prediction ratio is set to After each sample, the denominator of this value is reduced by one until it reaches The eviction prediction ratio no longer changes. The product of the eviction prediction ratio and the number of sampled objects equals the number of objects that can be evicted.

[0049] Step 4: Management of eviction candidates

[0050] Use the maximum heap structure and hash table to store prediction results. The management process is as follows:

[0051] Query: The top of the max heap represents the most distantly visited eviction candidate.

[0052] Added: When eviction candidates are first sampled and predicted, insert the predictions into the heap and hash table.

[0053] Modification: When the eviction candidate is sampled and predicted again, the new prediction result is directly inserted into the heap and the mapping value of the object in the hash table is updated to the new prediction value.

[0054] Deletion: It includes three cases: (1) When the farthest visited eviction candidate is evicted, delete the data at the top of the heap. And check whether the data in the heap is the same as the data in the hash table. If they are the same, remove the corresponding data in the hash table; otherwise, do not take any action on the hash table; (2) When an eviction candidate is accessed, delete the prediction result of the object from the hash table; (3) When the prediction model is updated, delete all the eviction candidate objects.

[0055] When retrieving data from the top of the heap, check whether the data of the object to be evicted at the top of the heap is equal to that in the hash table. If they are equal, the data evicts the object; otherwise, the data continues to obtain the top of the heap data for eviction.

Claims

1. A low-overhead, accurate learning-based web cache eviction strategy characterized by The following steps are involved: Step 1: Collection of training data Step 1.1: Set the sliding window size A method for dynamically adjusting the sliding window size based on a hyperparameter h is proposed. h is expressed as the ratio of objects in the sliding window to objects in the cache. The default value of h is 5. Step 1.2: Sample and label training data For each request, we randomly sample an object and record the sampling time. The sampling rules are as follows: (1) To improve the richness of the training data source, we randomly sample and record within the sliding window. (2) When we sample an object that has a record, we do not record the current information again. When a marked object is hit, a training sample is obtained and its record is deleted; when an object has not been requested and leaves the sliding window, a training sample is obtained, and its training label is its waiting time in the sliding window; after accumulating 65536 training samples, the model is updated; Step 2: Determine the prediction model Input features: The model input uses 34 input features, including 32 historical arrival time intervals, age, and object size; Prediction target: Predict the next arrival time interval of the object, take the logarithm of the predicted label; and determine the next arrival time of the object through the following formula: where t next Indicates the predicted next arrival time of the object. represents the predicted value of the model, t cur represents the current time, e is Napier's constant, and its value is approximately 2.718; Loss function: Use mean square error as the loss function, the formula is as follows: where y i represents the label of the i-th training sample, Represents the predicted value of the i-th sample; Step 3: Eviction of cached objects The objects that will be evicted by the offline optimal algorithm Belady are defined as unpopular objects. Therefore, in order to improve the performance of the cache algorithm, it is necessary to increase the sampling ratio of unpopular objects; Step 3.1: Sampling of eviction candidates Bidirectional sampling strategy; it includes: (1) directional reset sampling, sampling objects with the longest waiting time and the lowest number of accesses; when analyzing the data set using the Belady algorithm, it is found that objects with the longest waiting time and the lowest number of accesses are more likely to be evicted; (2) fast sampling, sampling newly arrived objects; based on the analysis of the public data set Twitter, the one-time hit miracle among newly arrived objects can be as high as 95%, and the one-time hit miracle object means that the object will not be accessed again after being accessed once; Step 3.1.1: Directed reset sampling At the beginning of each round of sampling, the cache queue is divided into 10 groups according to its length; therefore, sampling starts from the first group at the head of the queue, the sampling step is initially set to 1, and 256 objects are sampled each time; as the sampling pointer moves to the end of the queue, the frequency of object access will increase; therefore, compared with the previous group, the sampling step in each subsequent group is doubled, and the value sampling history request count is less than r max In addition, the number of requests encountered during the scan does not exceed r min The object will be sampled immediately; after completing 10 groups of sampling, the sampling pointer will be reset back to the head of the queue; Based on the cache queue object request frequency distribution, r min Set to the number of requests at its 10th percentile to sample objects with fewer accesses in groups 2-10; r max Set it to the 50th percentile of the number of requests to filter out objects with high request counts in groups 2-10; Step 3.1.2: Quick sampling Using an additional queue to record the IDs of newly arrived objects is called a record queue. Fast sampling samples newly arrived objects based on a first-in-first-out basis. After an object is sampled, it is removed from the record queue until the space occupied by the object in the record queue is less than 5% or the number of sampled objects exceeds 512. Once either of these two conditions is met, fast sampling ends. Step 3.2: Generation of eviction prediction ratio After each eviction, the prediction results of the candidates that were not evicted are recorded for subsequent evictions; You also need to set the eviction prediction ratio, which changes dynamically with each sampling; when there are no eviction candidates, the initial value of the eviction prediction ratio is set to After each sample, the denominator of this value is reduced by one until it reaches The eviction prediction ratio no longer changes; the product of the eviction prediction ratio and the number of sampled objects is equal to the number of objects that can be evicted; Step 4: Management of eviction candidates Use the maximum heap structure and hash table to store the prediction results; the management process is as follows: Query: The top of the max heap represents the farthest visited eviction candidate; Added: When eviction candidates are sampled and predicted for the first time, insert the prediction results into the heap and hash table; Modification: When sampling and predicting the eviction candidate again, the new prediction result is directly inserted into the heap, and the mapping value of the object in the hash table is updated to the new prediction value; Deletion: includes three cases: (1) When the farthest visited eviction candidate is evicted, delete the data at the top of the heap; and check whether the data in the heap is the same as the data in the hash table; if they are the same, remove the corresponding data in the hash table; otherwise, do not take any action on the hash table; (2) When an eviction candidate is accessed, delete the prediction result of the object from the hash table; (3) When the prediction model is updated, delete all the eviction candidate objects; When retrieving data from the top of the heap, check whether the data of the object to be evicted at the top of the heap is equal to that in the hash table; if they are equal, the data evicts the object; otherwise, the data continues to obtain the top of the heap data for eviction.

Images