Cache placement method, system and medium combining long and short time slots for optimization

Abstract

The present invention provides a cache placement method, system and medium combining long and short time slots for optimization, which relates to the field of computer communication technology, including: an edge cache server, the edge cache server including an HDD disk and an SSD disk, the HDD disk determines the popularity of video files, and then stores popular video files from the source station into the HDD disk, and the HDD disk will autonomously remove redundant unpopular files; the SSD disk stores high-heat files in the HDD disk; the method includes the following steps: step S1: performing a cache replacement of a long-period HDD disk; step S2: after completing the long-period HDD disk cache replacement, a short-term cache replacement is performed on the SSD disk. The present invention can improve the hit rate of the edge cache server, alleviate network congestion and transmission burden during peak request periods, reduce distribution delays, and improve user experience.

Description

Technical Field

[0001] The present invention relates to the field of computer communication technology, and in particular to a cache placement method, system and medium combining long and short time slots with optimization. Background Art

[0002] Content delivery network (CDN) is an effective technology for providing online video services. It guarantees the ability to serve a large number of user requests during peak hours by pre-caching popular videos in edge servers, while saving network traffic. However, in recent years, the demand for online video services has been growing explosively. A single server of online video websites such as iQiyi needs to process a large number of requests for millions of videos every day. In addition, the storage space and reading speed of edge servers are limited. Although traditional passive cache replacement schemes such as Least Recently Used (LRU) and Least Frequently Used (LFU) have the advantages of simplicity and efficiency, their cache hit rate can no longer meet actual needs. Therefore, the research hotspot in the cache field has gradually shifted to active caching and distribution strategies. The emergence and application of many prediction models show that excellent content popularity prediction models and cache replacement schemes can effectively improve the cache hit rate of CDN edge nodes, alleviate the transmission burden of CDN, improve the efficiency of end users receiving files, and greatly improve user experience.

[0003] It is understood that previous studies have not taken into account the actual reading speed of edge cache servers. In actual CDN servers, edge cache servers are composed of mechanical disks (HDD) and solid-state disks (SSD). HDD disks directly obtain video files from the source station. They have large capacity but slow reading speed. SSD disks read data from HDD disks. They have small capacity but fast reading speed and are used to store high-profile videos. Due to the limitation of reading speed, HDD disks are usually updated once a day or even a week, while SSD disks can be updated once a minute.

[0004] At present, there are very few studies on the combined optimization strategies for HDD and SSD read speeds designed to improve edge cache hit rates and suitable for actual CDN systems. Most of them do not consider the actual CDN composition, or only start from the perspective of improving disk service life. Therefore, it is necessary to design new cache replacement strategies to meet actual needs.

[0005] The invention patent with publication number CN107171961B discloses a caching method and device based on content popularity, including: dividing time into multiple time slots, the multiple time slots are the first k time slots of the current time slot, and the time slot length is T; learning the content objects of the first k time slots, obtaining the historical popularity of the content objects in the first k time slots, k is a natural number; predicting the popularity of the content objects in future time slots based on the historical popularity, and obtaining the predicted popularity; modeling the caching decision of the content objects as a 0-1 knapsack model based on the predicted popularity; solving the 0-1 knapsack model through a popularity priority greedy caching algorithm, obtaining the cache priority based on the solution result, and caching the content objects according to the cache priority.

[0006] Definitions of Abbreviations and Key Terms:

[0007] CDN Content Delivery Network Content Delivery Network

[0008] LRU Least Recently Used

[0009] LFU Least Frequently Used

[0010] HDD Hard Disk Drive

[0011] SSD Solid State Disk

[0012] MLR Multiple Linear Regression

[0013] RSC multiple linear regression model based on video age segmentation and clustering

[0014] VGP Video file Group Prediction Video file group prediction Summary of the invention

[0015] In view of the defects in the prior art, the present invention provides a cache placement method, system and medium combining long and short time slots for optimization.

[0016] According to a cache placement method, system and medium combining long and short time slots optimization provided by the present invention, the scheme is as follows:

[0017] In a first aspect, a cache placement method combining long and short time slot optimization is provided, the method comprising:

[0018] The edge cache server includes an HDD disk and an SSD disk. The HDD disk determines the popularity of video files, and then stores popular video files from the source station into the HDD disk. At the same time, the HDD disk will autonomously remove redundant unpopular files; the SSD disk stores high-temperature files in the HDD disk. The method includes the following steps:

[0019] Step S1: Perform cache replacement of long-cycle HDD disks;

[0020] Step S2: After completing the long-term HDD disk cache replacement, perform a short-term cache replacement on the SSD disk.

[0021] Preferably, the multivariate linear regression model RSC based on video age segmentation and clustering in step S1 is used as a long-term prediction algorithm, including two processes: offline preparation and online prediction;

[0022] The offline preparation process mainly extracts the historical request volume sequence of a certain length of each file and the relevant information tags including the release time, and then performs age grouping, clustering and linear fitting on the files.

[0023] Preferably, the step S1 specifically includes:

[0024] Step S1.1: Group by video age; group by release time of each file, and group files of similar age groups into the same group;

[0025] Step S1.2: Clustering based on similarity of popularity change trends; performing K-means sequence clustering on the files in each age group based on the historical request sequence, and clustering the time series with similar popularity change trends into one cluster; wherein, the Euclidean distance is used as the criterion for measuring the similarity of the time series, and the number of clusters is determined by the elbow rule;

[0026] Step S1.3: training model; after clustering is completed, each group contains multiple clusters, and a multivariate linear regression model is established for each cluster, and the request volume time series in the corresponding cluster is used for training. After the training is completed, each cluster obtains a set of fitting parameters;

[0027] Step S1.4: Test; according to the release time of the test file, calculate the Euclidean distance between the test file and several cluster centers in the age group corresponding to the test file, find the cluster center that is most similar to the popularity trend of the sequence, and then use the fitting parameters of the cluster cluster to predict the sequence and obtain the request volume of the next update cycle;

[0028] Step S1.5: Cache replacement: based on the predicted request volume ranking, the video with the highest predicted request volume is stored in the HDD disk, and the videos with low ranking are eliminated.

[0029] Preferably, in step S2, short-term cache replacement is performed on the SSD disk, and a video file group prediction VGP algorithm is used to classify and process the files mainly based on the request volume. Different types of files will use different judgment processes, including the following steps:

[0030] Step S2.1: Group the files according to their recent request status in the past few cycles. If they are requested multiple times in this period, they are classified as popular files; otherwise, they are classified as unpopular files.

[0031] Step S2.2: prediction: at each new moment, the number of requests for each file predicted in the long term minus the cumulative number of requests at the current moment can be predicted to predict the number of times each file will be requested until the next active update;

[0032] Step S2.3: Allocate popular files to the SSD disk. Although some unpopular files have not been frequently requested in the past, if they were requested at the previous moment, it is considered that the unpopular files have certain future request expectations and need to be allocated to the SSD disk;

[0033] Step S2.4: if there is still free space in the SSD disk after step S2.3 is completed, then according to the future request volume of each file predicted in step S2.2, several files with high future request volume and not currently in the SSD disk are allocated to the SSD disk;

[0034] If the SSD disk cache space overflows after step S2.3 is completed, several files with fewer future request volumes will be removed from the SSD disk based on the future request volumes of each file predicted in step S2.2.

[0035] In a second aspect, a cache placement system combining long and short time slots with optimization is provided, the system comprising:

[0036] Edge cache server, the edge cache server includes HDD disk and SSD disk, the HDD disk determines the popularity of video files, and then stores popular video files from the source station into the HDD disk, and the HDD disk will automatically remove unnecessary unpopular files; the SSD disk stores the high-temperature files in the HDD disk; the system includes the following modules:

[0037] Module M1: performs cache replacement of long-cycle HDD disks;

[0038] Module M2: After completing the long-term HDD disk cache replacement, perform a short-term cache replacement on the SSD disk.

[0039] Preferably, the multivariate linear regression model RSC based on video age segmentation and clustering in the module M1 is used as a long-term prediction algorithm, including two processes: offline preparation and online prediction;

[0040] The offline preparation process mainly extracts the historical request volume sequence of a certain length of each file and the relevant information tags including the release time, and then performs age grouping, clustering and linear fitting on the files.

[0041] Preferably, the module M1 specifically includes:

[0042] Module M1.1: Group by video age; group files according to their release time, and group files of similar age groups into the same group;

[0043] Module M1.2: Clustering based on similarity of popularity trends; K-means sequence clustering is performed on the files in each age group based on the historical request sequence, and time series with similar popularity trends are clustered into one cluster; Euclidean distance is used as the criterion for measuring the similarity of time series, and the number of clusters is determined by the elbow rule;

[0044] Module M1.3: Training model; after clustering is completed, each group contains multiple clusters. A multivariate linear regression model is established for each cluster, and the request volume time series in the corresponding cluster is used for training. After training, each cluster obtains a set of fitting parameters;

[0045] Module M1.4: Testing: Based on the release time of the test file, calculate the Euclidean distance between the test file and several cluster centers in the corresponding age group of the test file, find the cluster center that is most similar to the popularity trend of the sequence, and then use the fitting parameters of the cluster cluster to predict the sequence and obtain the request volume for the next update cycle;

[0046] Module M1.5: Cache replacement: Based on the predicted request volume ranking, the video with the highest predicted request volume is stored in the HDD disk, and the video with the lower ranking is eliminated.

[0047] Preferably, the module M2 performs short-term cache replacement for the SSD disk, adopts the video file group prediction VGP algorithm, and classifies the files mainly based on the request volume. Different types of files will adopt different judgment processes, including:

[0048] Module M2.1: Group files according to their recent request status. Files that are requested multiple times in this period are classified as popular files, otherwise they are classified as unpopular files.

[0049] Module M2.2: Prediction: At each new moment, the number of requests for each file at the long-term prediction minus the cumulative number of requests at the current moment can be used to predict the number of times each file will be requested until the next active update;

[0050] Module M2.3: Allocate popular files to the SSD disk. Although some unpopular files have not been frequently requested in the past, if they were requested at the previous moment, it is considered that the unpopular files have certain future request expectations and need to be allocated to the SSD disk;

[0051] Module M2.4: If there is still free space in the SSD disk after module M2.3 is completed, several files with high future request volumes and not currently in the SSD disk are allocated to the SSD disk according to the future request volumes of each file predicted by module M2.2;

[0052] If the SSD disk cache space overflows after module M2.3 is completed, several files with fewer future request volumes will be removed from the SSD disk based on the future request volumes of each file predicted by module M2.2.

[0053] According to a third aspect, a computer-readable storage medium storing a computer program is provided, wherein the computer program implements the steps of the method when executed by a processor.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] 1. The present invention takes into account the composition of the actual CDN edge cache system, and designs cache replacement algorithms for SSD disks and HDD disks based on their properties (cache size, reading speed and update cycle), thereby achieving combined optimization. Compared with the original passive or active algorithms, the present invention can improve the overall hit rate of edge servers and alleviate network congestion and transmission burden.

[0056] 2. The present invention improves the cache hit rate of the SSD disk as a whole, especially the hit rate during the peak request period. Compared with the traditional LRU and LFU, it can greatly reduce the content distribution delay and improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0058] Figure 1 This is a schematic diagram of the cache replacement method of an actual CDN server;

[0059] Figure 2 This is the main flow chart of the VGP algorithm;

[0060] Figure 3 It is the sub-flowchart of VGP algorithm;

[0061] Figure 4 This is an example diagram of the elbow rule;

[0062] Figure 5 This is a schematic diagram of the delay cache of the VGP algorithm;

[0063] Figure 6 A comparison of the average cache hit rate between the RSC model described in the embodiment of the present invention and four existing benchmark algorithms;

[0064] Figure 7 A comparison of the hit rate variation trend of the VGP model described in the embodiment of the present invention and two existing benchmark algorithms within one day;

[0065] Figure 8 A comparison of the average cache hit rate between the VGP model described in the embodiment of the present invention and two existing benchmark algorithms;

[0066] Fig. 9 A comparison of the hit rate variation trend of the combined optimization algorithm RSC+VGP described in an embodiment of the present invention and the existing benchmark algorithm within one day;

[0067] Fig.10 The figure is a comparison of the average cache hit rate between the combined optimization algorithm RSC+VGP described in the embodiment of the present invention and two existing benchmark algorithms. DETAILED DESCRIPTION

[0068] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0069] The embodiment of the present invention provides a cache placement method combining long and short time slots with optimization, which can be effectively applied to CDN systems. Figure 1 As shown, this application considers a single CDN edge cache server with limited cache size, which has an HDD disk and an SSD disk. The HDD disk uses a long time slot-active prediction algorithm to determine the popularity of video files, and then stores popular video files from the source station into the HDD disk. At the same time, the HDD disk will autonomously remove unnecessary unpopular files; for the SSD disk, a short time slot-passive replacement algorithm is designed to store high-temperature files in the HDD disk. The following steps are included:

[0070] Step S1: Perform cache replacement of the long-cycle HDD disk.

[0071] Step S2: After completing the long-term HDD disk cache replacement, perform a short-term cache replacement on the SSD disk.

[0072] Specifically, the cache of the HDD disk is replaced first. In summary, the present invention proposes a multivariate linear regression model (RSC) based on video age segmentation and clustering as a long-term prediction algorithm. The algorithm is divided into two processes: offline preparation and online prediction. The offline preparation process mainly extracts a certain length of historical request sequence of each file and information tags such as release time, and then performs age grouping, clustering and linear fitting on the files. It mainly includes the following steps:

[0073] Step S1.1: Group by video age. Generally speaking, the older the video is, the less likely it is that users will request it, and the lower the video popularity ranking is. The more recent the video is, the more likely users will request it. The long-term prediction algorithm first groups the files according to their release time, and groups files of similar age groups into the same group.

[0074] Step S1.2: Clustering based on similarity of popularity change trends; performing K-means sequence clustering on the files in each age group based on the historical request sequence, and clustering the time series with similar popularity change trends into one cluster; wherein, the Euclidean distance is used as the measure of time series similarity, and the number of clusters is determined by the elbow rule.

[0075] Step S1.3: training model; after clustering is completed, each group contains multiple clusters, and a multivariate linear regression model is established for each cluster, and the request volume time series in the corresponding cluster is used for training. After the training is completed, each cluster obtains a set of fitting parameters.

[0076] Step S1.4: Test; Based on the release time of the test file, calculate the Euclidean distance between the test file and several cluster centers in the age group corresponding to the test file, find the cluster center that is most similar to the popularity trend of the sequence, and then use the fitting parameters of the cluster cluster to predict the sequence to obtain the request volume for the next update cycle.

[0077] Step S1.5: Cache replacement: based on the predicted request volume ranking, the video with the highest predicted request volume is stored in the HDD disk, and the videos with low ranking are eliminated.

[0078] After completing the long-term HDD disk cache replacement, the short-term cache replacement is performed on the SSD disk. The present invention proposes a video file group prediction (VGP) algorithm. The algorithm mainly classifies files based on the request volume, and different types of files will use different judgment processes. It mainly includes the following steps:

[0079] Step S2.1: Group the files according to their recent request situations. If they are requested multiple times in this time period, they are classified as "hot" files; otherwise, they are classified as "cold" files.

[0080] Step S2.2: Prediction. At each new moment, the number of requests for each file predicted in the long term minus the cumulative number of requests at the current moment can be used to predict the number of times each file may be requested until the next active update. The predicted number of future requests can be used to roughly estimate the popularity difference of each file at the current moment.

[0081] Step S2.3: Allocate "hot" files to the SSD disk. Although some "unpopular" files have not been frequently requested in the past, if they were requested at the previous moment, it is considered that the "unpopular" files have certain future request expectations and also need to be allocated to the SSD disk.

[0082] Step S2.4: if there is still free space in the SSD disk after step S2.3 is completed, then according to the future request volume of each file predicted in step S2.2, several files with high future request volume and not currently in the SSD disk are allocated to the SSD disk;

[0083] If the SSD disk cache space overflows after step S2.3 is completed, several files with fewer future request volumes will be removed from the SSD disk based on the future request volumes of each file predicted in step S2.2.

[0084] Reference Figure 2 and Figure 3 As shown in the figure, the operation process of short-term cache replacement is shown in detail. The whole process consists of a main process nested in a sub-process, and the two processes operate on files from a global and individual perspective respectively.

[0085] Next, the present invention will be described in more detail.

[0086] The embodiment of the present invention provides a cache placement method combining long and short time slots with optimization, which is applicable to a single CDN edge cache server with limited cache size. Figure 1 As shown in Figure 1, the edge cache server has an origin server, an HDD disk, an SSD disk, and some end users. Assume that the set of files that users can request is F = {1, 2, ..., F}, all files in F are stored in the origin server, and the file sizes are normalized to 1. Define T SSD and T HDD Represent the update cycle size of SSD and HDD respectively, and T SSD <<T HDD In this example, we take T SSD For 1 minute, take T HDDAfter the user sends a request, the target file is first searched in the SSD disk. If it is a hit, it will be quickly distributed to the corresponding user terminal; if it is not a hit, it will be further searched in the HDD disk. If none of the edge servers hits it, the source station will directly send the file to the user.

[0087] For HDD disks, first divide the video collection into N groups (N>3) according to the release time. The time interval between groups can be from short to long, such as videos released within 1 month are Segment 1, videos released within 1 to 3 months are Segment 2, and videos released within 3 to 6 months are Segment 3, and so on.

[0088] Take the nth group of N groups of videos as an example, n∈{1,2,…,N}, and define the total number of videos that users can request in this group as M n , the video collection is The historical playback statistics time is T days, record video f i The number of requests on day j is v i,j ,j∈{1,2,…,T}, then the video f i The length of the playback time series V is T i,T It can be expressed as:

[0089] V i,T = {v i,1 ,v i,2 ,…,v i,T} (1)

[0090] Then, the K-means clustering algorithm is applied to each group to cluster the time series with similar request volume change trends into one category. Suppose two videos f are randomly selected from the video collection. p and f q , using Euclidean distance to measure the similarity between two time series:

[0091]

[0092] However, since the K-means algorithm needs to set the number of clusters K in advance, and the results obtained with different K values ​​may vary greatly, we choose to use the elbow rule to determine the K value. The elbow rule calculates the sum of the squared distance errors between the mass points of each cluster and the sample points within the cluster, and calls it the degree of distortion. The degree of distortion will decrease with the increase in categories, but for data with a certain degree of discrimination, the degree of distortion will be greatly improved when it reaches a certain critical point, and then slowly decrease. This critical point can be considered as a point with better clustering performance, and the elbow rule calls it the "elbow". Figure 4This is the curve of the variation of the distortion degree with the number of cluster centers when clustering videos released within one month. By observing the curve and using the elbow rule, the optimal number of clusters K can be locked. Obviously, in this figure, the optimal number of clusters is 6.

[0093] After clustering the N groups of data in the training set, we will get K1, K2, ..., K n ,…,K N Clusters and centroid time series. Considering that there is a strong linear correlation between the past and future popularity of video files, a multivariate linear regression (MLR) model is trained using the time series in each cluster, and the historical playback volume of the previous t days is used as input to predict the number of requests on the t+1 day. Define the use of the MLR model to get the predicted playback volume of the video on the t+1 day for:

[0094]

[0095] Among them, Θ n,k,t ={θ n,k,,1 ,θ n,k,2 ,…,θ n,k,t},k∈{1,2,…,K n} represents the model parameters obtained by training the time series in the kth cluster in the nth group.

[0096] Furthermore, the Euclidean distance between the time series of each group in the test set and its corresponding training set group is calculated to measure the similarity, and they are classified and predicted using the corresponding MLR model to obtain the predicted values ​​and rank them, so that videos with high popularity are stored and videos with low popularity are eliminated.

[0097] For SSD disks, for the sake of convenience, we assume that cache replacement is performed at the rth minute of day t+1. Short-term cache replacement emphasizes real-time performance. In the actual replacement process, due to the delay in algorithms and cache replacement operations, all operations need to be started before the end of the τth minute. When determining the files to be cached at the r+1th minute, only the request volume data from the r-1th minute and before can be used. For a specific schematic diagram, see Figure 5 First, an observation period T is set according to factors such as file request volume distribution and cache capacity. B , needs to satisfy 0<T B <T HDD Each time the cache of the SSD disk file is updated, first observe the time from the previous moment r-1 to the past T B The request volume distribution of each file during this period of time. Files that are requested multiple times during this period will be classified as "hot" files, otherwise they will be classified as "cold" files.

[0098] In addition, it is necessary to count the number of times each file has been requested since the last long-term cache replacement to the previous time r-1, using c i,r-1 represents the number of requests for the i-th file up to time r-1, based on the results of the long-term cache prediction algorithm It can be roughly estimated how many times the file will be requested before the next long-term cache replacement, that is, the remaining number of requests in the future

[0099]

[0100] After completing the file classification and estimating the future request volume, first store all the "hot" files in the cache. These files have been B Frequently requested within a time period, there is a high expectation of request volume. At the same time, although some "unpopular" items have not been frequently requested in the past, if they were requested within the previous time r-1, there is also a certain expectation of request volume and they also need to be stored in the cache.

[0101] After storing the above files with expected request volume in the cache, if there is still space left in the SSD disk cache, all the files not stored in the cache on the HDD disk will be cached according to the remaining request volume in the future. Ranking, according to the results, select several files with high future request volume to add to the cache; if the SSD disk cache space capacity overflows, all files in the SSD disk will be cached according to the remaining future request volume. Perform ranking and select several files with lower future request volume for elimination based on the results.

[0102] In order to further illustrate the technical effect of the present invention, the combined optimization method proposed in the embodiment of the present invention is simulated. We select cache hit rate as the evaluation criterion, and its calculation formula is formula (5), where L is the total number of predictions.

[0103]

[0104] For the simulation of the long-term algorithm RSC, four comparison algorithms were selected, namely LRU, Szabo-Huberman model (SH), multivariate linear regression model based on clustering (MLC), and multivariate linear regression model based on video age segmentation but without clustering (MLAS).

[0105] Figure 6 The simulation results of the long-term prediction algorithm RSC model and four benchmark methods are shown in Figure 2. The horizontal axis is the cache size of the HDD disk, which indicates the percentage of cacheable files to the total number of files, and the vertical axis is the average hit rate. In this simulation, L = 24. Figure 6It can be seen that the RSC model proposed by the present invention is more effective than the other four methods, and the MLC is second. This is because the clustering operation plays a certain role in distinguishing the popularity trends of different videos. The performance of MLAS is also better than that of traditional algorithms, which shows that the age of the video has an impact on the popularity of the video. The LFU and SH models have the worst effects. This is because it is not sufficient to only consider the playback volume of the previous day or to assume that the playback volume is in a simple linear relationship with the total playback volume before. It can also be seen from the figure that in the process of continuously increasing the cache size, although the hit rate has been increasing, the growth rate has gradually slowed down. This is because the number of popular videos is limited, and unpopular videos actually account for the majority, so the hit rate cannot be increased by simply increasing the cache size.

[0106] For the short-time algorithm VGP simulation, the traditional 2Q algorithm and LRU-LFU algorithm are selected as comparison models.

[0107] Figure 7 and Figure 8 For the simulation results of the VGP model of the short-term prediction algorithm running alone, the cache hit rate is also selected as the evaluation criterion. Since the hit rate is actually a statistical probability, if the amount of data is too small during calculation, it is easy to produce errors due to randomness. Therefore, in the simulation analysis of short-term request prediction, the average hit rate is calculated every 5 update cycles (minutes), and the overall hit rate of all requests and the total hit rate during the peak period of daily requests are also calculated. Assuming that the storage capacity of the cache is 20%, that is, it can store up to 20% of the videos in all files. At this time, the corresponding hit rate obtained by each prediction algorithm changes with the time of the day as shown in the following curve. Figure 7 As shown in the figure. First, it can be clearly observed that the performance of VGP and 2Q algorithms is much better than LRU-LFU. Second, the request hit rate during the peak period (20:00-22:00) is an important indicator for evaluating the edge server cache solution. Figure 7 It shows that the number of video file requests gradually reaches its peak from the afternoon to late at night, and the advantage of the VGP algorithm over the 2Q algorithm is more obvious at this time.

[0108] Figure 8The overall hit rate of various algorithms in one day is shown as the SSD cache size changes from 0% to 100%. It can be seen that when the cache capacity gradually increases, the hit rate of each algorithm simulation also gradually increases. When the storage capacity is 100%, it means that all video files can be stored in the cache without overflow. It can be observed that at this time, both the VGP and LRU-LFU algorithms can achieve a 100% hit rate. However, since the 2Q algorithm divides the cache space into two parts, some video files will be unnecessary removed during the caching process, resulting in failure to achieve a 100% hit rate. From the results in the figure, it can be seen that the VGP algorithm has better performance than the traditional 2Q and LRU-LFU algorithms because it has a flexible classification of video files that can be adjusted at any time.

[0109] Finally, we focus on the performance of each short-term prediction algorithm after combining it with the long-term prediction algorithm. It should be noted here that since the storage content of the SSD disk is included in the server as a whole (HDD disk), the hit rate of the SSD disk can never exceed that of the HDD disk. Taking the request data of the server for a whole day as a sample, Fig. 9 When the storage capacity of the HDD disk is set to 50% of all files and the cache size of the SSD disk is set to 10% of the HDD disk, the corresponding hit rate obtained by each prediction algorithm changes with the time of day. It can be seen that during the peak request period, the performance of the combined algorithm RSC+VGP proposed by the present invention is significantly better than 2Q and LRU-LFU.

[0110] Fig.10 The hit rate of each disk in the server changes with the overall storage capacity of the server (HDD disk) when the storage capacity of the SSD disk accounts for 15% of the total storage capacity. The SSD disk hit rate curve obtained by the simulation of each short-term prediction algorithm will gradually approach the HDD disk curve.

[0111] The present invention takes into account the composition of the actual CDN edge cache system, and designs cache replacement algorithms for SSD and HDD disks based on their properties (cache size, read speed, and update cycle), thus achieving combined optimization. Compared with the original passive or active algorithms, it can improve the overall hit rate of edge servers and alleviate network congestion and transmission burden. Figure 6 , 9 , as shown in Figure 10.

[0112] In addition, the combined optimization method proposed in the present invention improves the cache hit rate of the SSD disk, especially the hit rate during the peak request period (20:00-21:00). Compared with the traditional LRU and LFU, it can greatly reduce the content distribution delay and improve the user experience. Figure 7 , 8 The simulation results are shown.

[0113] The embodiments of the present invention provide a cache placement method, system and medium that optimizes long and short time slots. Factors such as the cache architecture, update cycle and operating performance of edge cache servers in an actual CDN network are taken into consideration. Only a small amount of necessary information is extracted from the server request log for processing to achieve cache replacement that combines long and short time slots. Compared with replacement algorithms such as LRU and LFU commonly used in current CDN servers, the system performance is improved.

[0114] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0115] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A cache placement method combining long and short time slot optimization, characterized in that: include: The edge cache server includes an HDD disk and an SSD disk. The HDD disk determines the popularity of video files, and then stores popular video files from the source station into the HDD disk. At the same time, the HDD disk will autonomously remove redundant unpopular files; the SSD disk stores high-temperature files in the HDD disk. The method includes the following steps: Step S1: Perform cache replacement of long-cycle HDD disks; Step S2: After completing the long-term HDD disk cache replacement, perform a short-term cache replacement on the SSD disk; In the step S1, the multivariate linear regression model RSC based on video age segmentation and clustering is used as a long-term prediction algorithm, including two processes: offline preparation and online prediction; The offline preparation process mainly extracts the relevant information tags including the historical request volume sequence of a certain length and the release time of each file, and then performs age grouping, clustering and linear fitting on the files; The step S1 specifically includes: Step S1.1: Group by video age; group by release time of each file, and group files of similar age groups into the same group; Step S1.2: Clustering based on similarity of popularity change trends; performing K-means sequence clustering on the files in each age group based on the historical request sequence, and clustering the time series with similar popularity change trends into one cluster; wherein, the Euclidean distance is used as the criterion for measuring the similarity of the time series, and the number of clusters is determined by the elbow rule; Step S1.3: training model; after clustering is completed, each group contains multiple clusters, and a multivariate linear regression model is established for each cluster, and the request volume time series in the corresponding cluster is used for training. After the training is completed, each cluster obtains a set of fitting parameters; Step S1.4: Test; according to the release time of the test file, calculate the Euclidean distance between the test file and several cluster centers in the age group corresponding to the test file, find the cluster center that is most similar to the popularity trend of the sequence, and then use the fitting parameters of the cluster cluster to predict the sequence and obtain the request volume of the next update cycle; Step S1.5: Cache replacement: based on the predicted request volume ranking, the video with the highest predicted request volume is stored in the HDD disk, and the video with the lower ranking is eliminated; In step S2, short-term cache replacement is performed on the SSD disk, and the video file group prediction VGP algorithm is used to classify the files mainly based on the request volume. Different types of files will use different judgment processes, including the following steps: Step S2.1: Group the files according to their recent request status in the past few cycles. If they are requested multiple times in this period, they are classified as popular files; otherwise, they are classified as unpopular files. Step S2.2: prediction: at each new moment, the number of requests for each file predicted in the long term minus the cumulative number of requests at the current moment can be predicted to predict the number of times each file will be requested until the next active update; Step S2.3: Allocate popular files to the SSD disk. Although some unpopular files have not been frequently requested in the past, if they were requested at the previous moment, it is considered that the unpopular files have certain future request expectations and need to be allocated to the SSD disk; Step S2.4: if there is still free space in the SSD disk after step S2.3 is completed, then according to the future request volume of each file predicted in step S2.2, several files with high future request volume and not currently in the SSD disk are allocated to the SSD disk; If the SSD disk cache space overflows after step S2.3 is completed, several files with fewer future request volumes will be removed from the SSD disk based on the future request volumes of each file predicted in step S2.

2.

2. A cache placement system combining long and short time slots with optimization, characterized in that: include: Edge cache server, the edge cache server includes HDD disk and SSD disk, the HDD disk determines the popularity of video files, and then stores popular video files from the source station into the HDD disk, and the HDD disk will automatically remove unnecessary unpopular files; the SSD disk stores the hottest files in the HDD disk; the system Includes the following modules: Module M1: performs cache replacement of long-cycle HDD disks; Module M2: After completing the long-term HDD disk cache replacement, perform short-term cache replacement on the SSD disk; The multivariate linear regression model RSC based on video age segmentation and clustering in the module M1 is used as a long-term prediction algorithm, including two processes: offline preparation and online prediction; The offline preparation process mainly extracts the historical request volume sequence of a certain length of each file and the relevant information tags including the release time, and then performs age grouping, clustering and linear fitting on the files; The module M1 specifically includes: Module M1.1: Group by video age; group files according to their release time, and group files of similar age groups into the same group; Module M1.2: Clustering based on similarity of popularity trends; K-means sequence clustering is performed on the files in each age group based on the historical request sequence, and time series with similar popularity trends are clustered into one cluster; Euclidean distance is used as the criterion for measuring the similarity of time series, and the number of clusters is determined by the elbow rule; Module M1.3: Training model; after clustering is completed, each group contains multiple clusters. A multivariate linear regression model is established for each cluster, and the request volume time series in the corresponding cluster is used for training. After training, each cluster obtains a set of fitting parameters; Module M1.4: Testing: Based on the release time of the test file, calculate the Euclidean distance between the test file and several cluster centers in the corresponding age group of the test file, find the cluster center that is most similar to the popularity trend of the sequence, and then use the fitting parameters of the cluster cluster to predict the sequence and obtain the request volume for the next update cycle; Module M1.5: Cache replacement: based on the predicted request volume ranking, the video with the highest predicted request volume is stored in the HDD disk, and the video with the lower ranking is eliminated; The module M2 performs short-term cache replacement for the SSD disk and adopts the video file group prediction VGP algorithm to classify the files mainly based on the request volume. Different types of files will adopt different judgment processes, including: Module M2.1: Group files according to their recent request status. Files that are requested multiple times in this period are classified as popular files, otherwise they are classified as unpopular files. Module M2.2: Prediction: At each new moment, the number of requests for each file at the long-term prediction minus the cumulative number of requests at the current moment can be used to predict the number of times each file will be requested until the next active update; Module M2.3: Allocate popular files to the SSD disk. Although some unpopular files have not been frequently requested in the past, if they were requested at the previous moment, it is considered that the unpopular files have certain future request expectations and need to be allocated to the SSD disk; Module M2.4: If there is still free space in the SSD disk after module M2.3 is completed, several files with high future request volumes and not currently in the SSD disk are allocated to the SSD disk according to the future request volumes of each file predicted by module M2.2; If the SSD disk cache space overflows after module M2.3 is completed, several files with fewer future request volumes will be removed from the SSD disk based on the future request volumes of each file predicted by module M2.

2.

3. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to claim 1 are implemented.

Images