Management system compatible with advanced caching algorithm in content delivery network (CDN)

Abstract

The invention discloses a management system compatible with an advanced cache algorithm in a content distribution network CDN. The management system comprises an Inode table, an algorithm library, a multi-stage FIFO queue, a controller and a recovery pool, the Inode table is a hash table stored in a memory and is used for quickly associating key information of each data object; the algorithm library comprises a plurality of cache strategies based on lists, and the cache strategies are processed as priority queues; the multi-level FIFO queues construct a layered storage structure, and each FIFO queue is used for managing a storage device strip; the controller is based on an in-memory cache simulator, and the working process comprises a preprocessing stage and a real-time adjustment stage; and the recycling pool is used for storing and managing the virtual stripes and the physical stripes. According to the system, the problems of write amplification and significant overhead caused when a traditional cache algorithm is applied to persistent storage can be solved, the requirement of high hit rate can be considered, and the I / O overhead can be effectively reduced.

Description

Technical Field

[0001] This invention relates to the field of network transmission technology, and in particular to a management system compatible with advanced caching algorithms in a content delivery network (CDN). Background Technology

[0002] With the rise of cloud computing and the explosive growth of data volume, people have begun to consider using caching technologies to improve the efficiency of large-scale data storage systems, which typically rely on persistent storage devices such as hard drives and solid-state drives (SSDs). By leveraging geographical advantages, such as setting up a Content Delivery Network (CDN), data access speeds can be accelerated. In the CDN architecture, the caching system plays a central role. When a client's request is sent to a CDN node, the system first checks if a cached copy of the requested resource exists locally. If it does, it sends it directly to the client, avoiding the need to request data from the origin server; otherwise, it requests the data from the origin server. Effective caching strategies not only reduce data transmission latency but also alleviate the load on the origin server; therefore, cache hit rate has become a key metric for evaluating CDN performance.

[0003] Based on extensive experimental and theoretical analysis, the First-In-First-Out (FIFO) algorithm typically has a relatively low hit rate, making it seem like an unsuitable choice. However, the development of the Internet of Things (IoT) has led to a massive number of small files (typically less than 1MB). The widespread existence of these small files makes random small write operations common, which can not only shorten the lifespan of storage devices but also affect write performance. Therefore, in practical file system caching applications, sequential merge writes are often used to handle these small write operations to improve efficiency. In this case, while sequential write systems using advanced caching algorithms—such as Least Recently Used (LRU) and Segmented Least Recently Used (SLRU)—can improve the hit rate, they also introduce additional I / O overhead due to the basic operation of boosting the relative position of accessed objects in advanced caching algorithms. For example, in the Least Recently Used (LRU) algorithm, if the hit rate is 90%, the boosting operation required for each hit can actually lead to a 10-fold write amplification effect. In contrast, the First-In-First-Out (FIFO) algorithm avoids this problem due to its simple management mechanism. Therefore, the problem that existing technologies need to solve is how to find a balance between the low I / O overhead and low hit rate brought by FIFO and the high hit rate and high I / O overhead of advanced caching algorithms. Summary of the Invention

[0004] The purpose of this invention is to provide a management system compatible with advanced caching algorithms in a content delivery network (CDN). This system can solve the write amplification and significant overhead problems caused by the application of traditional caching algorithms (such as LRU and LFU) on persistent storage (such as HDD and SSD), and can both meet the requirements of high hit rate and effectively reduce I / O overhead.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A management system compatible with advanced caching algorithms in a Content Delivery Network (CDN), the system comprising an Inode table, an algorithm library, a multi-level FIFO queue, a controller, and a recycling pool, wherein:

[0007] An inode table is a hash table stored in memory used to quickly associate key information of each data object, including the object ID, its location on the storage device, the object size, and the virtual stripe ID to which it belongs. When a data object is found, its location is obtained based on the index in the inode table without any additional file system operations.

[0008] The algorithm library includes multiple list-based caching strategies, which are treated as priority queues. Whenever a new request object arrives, the priority of the request object is adjusted according to the caching strategies in the algorithm library, and the relative position of the request object in the queue is determined by other modules of the system.

[0009] The multi-level FIFO queues construct a hierarchical storage structure. Each FIFO queue is used to manage storage device stripes, including stripe ingress, stripe egress, and a set of write buffers called memstripes. New data is first written to the write buffer of a specific FIFO according to the priority of the requesting object. Once the write buffer is full, a stripe is selected from the recycling pool to store the overflow data and added to the FIFO queue.

[0010] The controller is based on an in-memory cache simulator, and its operation includes a preprocessing stage and a real-time adjustment stage; among which:

[0011] In the preprocessing stage: the controller extracts key features by analyzing historical cache access data. Key features include the size distribution of requests, access frequency, and spatiotemporal locality between requests.

[0012] During the real-time adjustment phase: The controller continuously monitors the cache hit rate and compares the monitored hit rate with the expected hit rate. When a significant difference is detected, the controller will automatically adjust the length and number of cache queues and the number of virtual stripes in the queues.

[0013] The recycling pool is used to store and manage virtual and physical stripes. When the number of stripes processed by the multi-level FIFO queue reaches the capacity limit, the stripe is pushed to the recycling pool. At this time, the data on the stripe recycled by the recycling pool is not deleted immediately. The data in the stripe is only cleared before the stripe is about to be reused and overwritten.

[0014] As can be seen from the technical solution provided by the present invention, the above system can solve the write amplification and significant overhead problems caused by the application of traditional caching algorithms (such as LRU and LFU) on persistent storage (such as HDD and SSD), and can both meet the requirements of high hit rate and effectively reduce I / O overhead. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of a management system architecture compatible with advanced caching algorithms in a content delivery network (CDN) provided in an embodiment of the present invention;

[0017] Figure 2 This is a schematic diagram illustrating the operation process of the management system described in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments, and do not constitute a limitation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0019] like Figure 1 The diagram shows a management system architecture for a content delivery network (CDN) compatible with advanced caching algorithms, as provided in this embodiment of the invention. The system includes an Inode table, an algorithm library, a multi-level FIFO queue, a controller, and a recycling pool, wherein:

[0020] An inode table is a hash table stored in memory used to quickly associate key information for each data object, including the object ID, its location on the storage device, its size, and its virtual stripe ID (vstripe ID). This design makes data retrieval fast and efficient, maintaining low latency even under high-frequency access. When a data object is found, its location is obtained based on the index in the inode table, without additional file system operations. Conversely, if a data object is not found, the system immediately knows that the data is not on disk, allowing for rapid subsequent processing.

[0021] The algorithm library includes several list-based caching strategies, such as LRU and Least Frequently Used (LFU), and treats these caching strategies as priority queues. Whenever a new request object arrives, the priority of the request object is adjusted according to the caching strategy in the algorithm library, and the relative position of the request object in the queue is determined by the system's Inode table.

[0022] A multi-level FIFO queue constructs a hierarchical storage structure. Each FIFO queue manages storage device stripes, including stripe ingress, stripe egress, and a set of write buffers called memstripes. New data is first written to the write buffer of a specific FIFO according to the priority of the requesting object. Once the write buffer is full, a stripe is selected from the recycling pool to store the overflow data and added to the FIFO queue. The multi-level FIFO queue design allows the system to quickly process a large number of writes to non-hot resources, while avoiding the premature eviction of hot data in higher-level FIFO queues. In the specific implementation, this multi-level FIFO queue design is used to approximate advanced caching algorithms, and the multi-level FIFO queues are dynamically configured online based on the training results in the offline stage, enabling the system of this application to exhibit good adaptability to different cache eviction algorithms and workloads.

[0023] The controller is based on an in-memory cache simulator, and its operation includes a preprocessing stage and a real-time adjustment stage; among which:

[0024] In the preprocessing stage: the controller extracts key features by analyzing historical cache access data. These key features include the size distribution of requests, access frequency, and spatiotemporal locality between requests. These key features are used to form a preliminary understanding to guide the initial setting of cache configuration.

[0025] During the real-time adjustment phase: The controller continuously monitors the cache hit rate and compares the monitored hit rate with the expected hit rate. When a significant difference is detected, the controller will automatically adjust the length and number of cache queues and the number of virtual stripes (vstripes) in the queues. This adjustment is based on the current cache usage pattern and I / O load to ensure the effective utilization of cache resources.

[0026] The recycling pool is used to store and manage virtual stripes (vstripe) and physical stripes (pstripe). When the number of stripes processed by the multi-level FIFO queue reaches the capacity limit, the stripe is pushed to the recycling pool. At this time, the data on the stripe reclaimed by the recycling pool is not deleted immediately. The data in the stripe is only cleared before the stripe is about to be reused and overwritten. This mechanism not only ensures the security of the data, but also optimizes the use of storage space.

[0027] Based on the management system, when a request is received, the management system will first check the inode table to find the required data item;

[0028] If the search is successful, then perform the update operation;

[0029] If the search fails and the storage space is full, the management system will delete some old data before adding new data; if there is still enough storage space, new data will be added directly.

[0030] like Figure 2 The diagram shown is a schematic representation of the operation process of the management system according to an embodiment of the present invention. The specific operation flow of the management system is as follows:

[0031] Add operation: If data is not found in the cache and needs to be added, first, the position of the data in the cache priority queue is evaluated (depending on the selected algorithm, the newly added data may not necessarily be in the lowest priority queue), and the virtual stripe ID that is closest to it is found; the data is then stored in the write buffer of the FIFO queue of the corresponding virtual stripe; when the write buffer capacity reaches the limit, the pwrite function is used to write the data to the corresponding location on the disk, and a new entry for the data is created in the innode table with the offset of the current write position and the virtual stripe ID set to -1 to indicate that this is newly written data;

[0032] Update operation: If data is requested again in the cache, the position of the data in the priority queue needs to be updated. First, the new rank of the data is determined according to the caching algorithm. Then, the virtual stripe ID that is closest to this new rank is calculated, and the virtual stripe ID is updated to the corresponding entry in the Inode table. This process ensures that the position of the data in the cache reflects its latest access frequency and importance.

[0033] Deletion operation: When the write buffer capacity of a FIFO queue reaches its limit and space is needed to store new data, the old data will be deleted. First, the data stored in the write buffer that is about to be overwritten is searched, and their virtual stripe IDs are checked. If the virtual stripe ID is -1 or is in a lower priority queue, the data will be deleted and removed from the innode table, and the new data will directly overwrite the old data. If the virtual stripe ID of the data is in a higher priority queue, it will be rewritten to the write buffer of the corresponding virtual stripe queue. In addition, when a virtual stripe needs to be added back to the queue, the virtual stripe is placed at the front of the queue because it does not occupy actual disk space.

[0034] It is worth noting that the contents not described in detail in the embodiments of the present invention belong to the prior art known to those skilled in the art.

[0035] In its implementation, the management system described in this application (named BOSS+) was evaluated using traces from leading domestic and international CDNs on both SSDs and HDDs. The results showed that compared to a heavily loaded state-of-the-art (SOTA) storage system, the hit rate of the management system described in this application was 17% higher than the FIFO baseline, and the average request response latency was reduced by 34%. Therefore, the management system described in this application can solve the write amplification and significant overhead problems caused by traditional caching algorithms (such as LRU and LFU) when applied to persistent storage (such as HDDs and SSDs), balancing the need for high hit rates with effective reduction of I / O overhead.

[0036] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims. The information disclosed in the background section is intended only to enhance the understanding of the overall background technology of the present invention and should not be construed as an admission or implication in any way that such information constitutes prior art known to those skilled in the art.

Claims

1. A management system compatible with advanced caching algorithms in a Content Delivery Network (CDN), characterized in that, The system comprises an inode table, an algorithm library, a multi-level FIFO queue, a controller and a recycling pool, wherein: The inode table is a hash table stored in memory, which is used to quickly associate the key information of each data object, including object ID, location on storage device, object size and virtual stripe ID to which the object belongs; when a data object is hit, the location of the data object is obtained according to the index in the inode table, without additional file system operations; The algorithm library comprises a plurality of list-based cache strategies, and the cache strategies are processed as a priority queue; whenever a new request object arrives, the priority of the request object is adjusted according to the cache strategies in the algorithm library, and the relative position of the request object in the queue is determined by other modules of the system; The multi-level FIFO queue constructs a hierarchical storage structure, each FIFO queue is used to manage a storage device stripe, including a stripe inlet, a stripe outlet and a group of write buffers called memstripes; new data is first written into the write buffer of a specific FIFO according to the priority of the request object; once the write buffer is full, a stripe is selected from the recycling pool to store the overflow data, and the stripe is added to the FIFO queue; The controller is based on an in-memory cache simulator, and the working process comprises a preprocessing stage and a real-time adjustment stage; wherein: In the preprocessing stage: the controller extracts key features by analyzing historical cache access data, including the size distribution of requests, access frequency and spatiotemporal locality between requests; In the real-time adjustment stage: the controller continuously monitors the hit rate of the cache, and compares the monitored hit rate with the expected hit rate; when a significant difference is detected, the controller will automatically adjust the length and number of the cache queue and the number of virtual stripes in the queue; The recycling pool is used to store and manage virtual stripes and physical stripes; when the number of stripes processed by the multi-level FIFO queue reaches the upper limit of the capacity, the stripe is pushed to the recycling pool; at this time, the data on the stripe recycled by the recycling pool is not immediately deleted, but only when the stripe is about to be reused and overwritten, the data in the stripe is cleared.

2. The management system for compatible advanced cache algorithm in a content delivery network (CDN) according to claim 1, wherein, Based on the management system, when a request is received, the management system will first check the inode table to find the required data item; If the search is successful, an update operation is performed; If the search fails and the storage space is full, the management system will delete some old data before adding new data; if there is enough storage space, the new data is directly added.

3. The management system of claim 2, wherein, The specific operation process of the management system is as follows: Add operation: if a data is not found in the cache, it needs to be added, first evaluate the position of the data in the cache priority queue, and find the virtual stripe ID similar to it; the data is then stored in the write buffer of the FIFO queue of the corresponding virtual stripe; when the capacity of the write buffer reaches the upper limit, use the pwrite function to write the data to the corresponding position on the disk, and create a new entry for the data in the inode table, with the offset being the current write position and the virtual stripe ID being -1, indicating that it is newly written data; Update operation: If a data is requested again in the cache, the position of the data in the priority queue needs to be updated. First, the new rank of the data is determined according to the cache algorithm. Then, the virtual stripe ID closest to the new rank is calculated, and the virtual stripe ID is updated to the corresponding entry in the Inode table. Delete operation: When the capacity of the write buffer in a FIFO queue reaches the upper limit and space is needed to store new data, old data will be deleted. First, find the data stored in the write buffer that will be overwritten, and check their virtual stripe IDs. If the virtual stripe ID is -1 or in a lower priority queue, the data will be deleted and removed from the Inode table, and the new data will directly overwrite the old data. If the virtual stripe ID of the data is in a higher priority queue, it will be re-written to the write buffer in the corresponding virtual stripe queue. In addition, when a virtual stripe needs to be re-added to the queue, the virtual stripe is placed directly at the front of the queue.

Images