Method and computing device for memory cache system

Abstract

The present invention provides a method and computing device for a memory cache system, wherein the memory cache system is used to store key-value objects, and the method comprises: creating a cache instance according to a specified cache capacity and an estimated number of objects, wherein the cache instance comprises a small queue and a main queue, wherein the small queue is used to store objects that are recent or have an access frequency lower than an access threshold, and the main queue is used to store objects whose access frequency is equal to and / or greater than the access threshold; writing the objects to be cached to the cache instance, and always performing a write operation to the small queue; wherein, when the cache capacity is insufficient, the cache key-value objects are preferentially eliminated from the small queue. According to the technical solution of the present invention, a high cache hit rate can be guaranteed, cache pollution can be reduced, memory security and concurrency can be guaranteed, and the development and maintenance of the cache system can be simplified.

Description

Technical Field

[0001] The present invention relates to the technical field of software development, and in particular to a method and computing device for an efficient memory cache system. Background Art

[0002] With the rapid development of the Internet, Web applications are facing increasing performance challenges. Demands such as high concurrency, massive data, and low latency have put forward higher requirements for traditional cache systems. Existing cache algorithms such as the LRU algorithm (least recently used algorithm) are simple to implement, but have low hit rates and high memory consumption under certain workloads.

[0003] The traditional LFU algorithm (least frequently used algorithm) needs to maintain the access frequency counter of each object, which has a high space complexity. As the number of cached objects increases, the memory occupied by the frequency counter will increase sharply, resulting in excessive system memory pressure. The traditional LFU algorithm eliminates objects based on the lowest access frequency. In some scenarios, this strategy may cause cache pollution problems. For example, for scanning access patterns, LFU will keep a large number of one-time access objects in the cache for a long time, occupying the space of hot objects and reducing the overall cache hit rate.

[0004] To this end, a technical solution is needed that can ensure a high cache hit rate, reduce cache pollution, ensure memory safety and concurrency, and simplify the development and maintenance of the cache system. Summary of the invention

[0005] The present invention aims to provide a method and computing device for an efficient memory cache system, which can solve the data overflow problem during chip convolution calculation, improve the computing efficiency of the chip, and reduce the computing amount of the chip.

[0006] According to one aspect of the present invention, a method for a memory cache system is provided, wherein the memory cache system is used to store key-value objects, and the method comprises:

[0007] Create a cache instance according to the specified cache capacity and the estimated number of objects, the cache instance including a small queue and a main queue, the small queue is used to store objects that are recently accessed or have a frequency of access lower than an access threshold, and the main queue is used to store objects that have a frequency of access equal to and / or greater than the access threshold;

[0008] Write the cached object to the cache instance, and always write to the small queue;

[0009] When the cache capacity is insufficient, cache key-value objects are eliminated from the small queue first.

[0010] According to some embodiments, writing the object to be cached to the cache instance includes:

[0011] Call the read method to query the corresponding value according to the key of the object to be cached, wherein if the key exists, the corresponding value is returned and the access frequency of the object to be cached is increased; if the key does not exist, a null value is returned.

[0012] According to some embodiments, when the number of access frequencies of the cached key-value object is greater than the access threshold, the cached key-value object is moved from the small queue to the main queue.

[0013] According to some embodiments, when the cache capacity is insufficient, the cache key-value objects are preferentially eliminated from the small queue, including:

[0014] The cache key-value objects whose access frequency is lower than the frequency elimination threshold are preferentially eliminated from the small queue.

[0015] According to some embodiments, when the cache capacity is insufficient, the cached key-value objects are preferentially eliminated from the small queue, including:

[0016] If the small queue is empty, the cache key-value object is eliminated from the main queue.

[0017] According to some embodiments, when the small queue is empty, the cache key-value objects whose access frequency is lower than a frequency elimination threshold and / or whose residence time is lower than a residence time elimination threshold are eliminated.

[0018] According to some embodiments, when reading the cached key-value object, the corresponding object metadata is updated, and the object metadata includes the access frequency and the queue to which it belongs.

[0019] According to some embodiments, the write and eviction policies of the memory cache system are dynamically adjusted.

[0020] When the workload of the memory cache system changes, the capacity of the small queue and the main queue, and / or the residence time and / or access frequency requirements of the cache key-value objects are adaptively adjusted.

[0021] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the method as described above is implemented.

[0022] According to another aspect of the present invention, there is provided a computing device, comprising:

[0023] Processor; and

[0024] A memory stores a computer program, and when the computer program is executed by the processor, the method described in any one of the above items is implemented.

[0025] According to an embodiment of the present invention, a method for a high-efficiency cache system is provided, which aims to improve the cache hit rate and resource utilization efficiency in a multi-threaded environment. The present invention combines a variety of data structures and algorithms. After creating a cache instance according to the specified cache capacity and the estimated number of objects, the write method is called to write the cache object into the cache. When the cache capacity is insufficient, objects with low access frequency are automatically eliminated according to the algorithm to make cache space for new cache objects. Through flexible cache management strategies and optimized concurrent processing mechanisms, while ensuring high performance, memory overhead and cache replacement loss are minimized. The present invention can ensure a high cache hit rate, reduce cache pollution, ensure memory security and concurrency, and simplify the development and maintenance of the cache system.

[0026] According to some embodiments, by setting an upper limit on the cache capacity, unlimited cache expansion can be avoided, ensuring that system resources are reasonably controlled and efficiently used. At the same time, the elimination strategy based on access frequency can give priority to retaining popular objects, improve the cache hit rate, and reduce frequent access to backend data sources, thereby improving the response speed and performance of the overall system.

[0027] According to some embodiments, the strategy can dynamically adjust the cache content according to the real-time access pattern and cache status, so that the cache is closer to the needs of users or applications. When new objects are added to the cache, objects with low access frequency are eliminated to ensure the freshness and relevance of the cache content.

[0028] According to some embodiments, the automated elimination mechanism reduces the need for manual intervention, reduces the complexity and cost of cache management, and at the same time, since the selection and elimination of cache objects are automatically completed based on algorithms, performance bottlenecks or resource waste caused by human judgment errors can be reduced.

[0029] According to some embodiments, according to different application scenarios and requirements, the cache capacity can be flexibly adjusted, different elimination algorithms can be selected, and even multiple strategies can be combined to achieve the best cache effect.

[0030] According to some embodiments, frequently accessed data is quickly provided, latency is reduced, and front-end response speed is increased, thereby significantly improving user experience. This is particularly important in applications with high concurrency or high real-time requirements.

[0031] It is to be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for describing the embodiments are briefly introduced below.

[0033] Figure 1A flow chart of a method for an efficient memory cache system according to an example embodiment is shown.

[0034] Figure 2 A schematic diagram of a working process of a memory cache system according to an example embodiment is shown.

[0035] Figure 3 A schematic diagram showing a process of removing a queue object according to an example embodiment.

[0036] Figure 4 A block diagram of a computing device is shown according to an exemplary embodiment. DETAILED DESCRIPTION

[0037] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that the present invention will be comprehensive and complete and fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0038] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, those skilled in the art will appreciate that the technical solution of the present invention can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present invention.

[0039] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0040] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0041] It should be understood that although the terms first, second, third, etc. may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another component. Therefore, the first component discussed below can be referred to as the second component without departing from the teachings of the present inventive concept. As used herein, the term "and / or" includes any one of the associated listed items and all combinations of one or more.

[0042] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0043] Those skilled in the art will appreciate that the drawings are merely schematic diagrams of example embodiments, and the modules or processes in the drawings are not necessarily necessary for implementing the present invention, and therefore cannot be used to limit the protection scope of the present invention.

[0044] Although existing memory cache systems have significant advantages in improving data access speed and system performance, they also have some inherent disadvantages and challenges. The smaller capacity of memory compared to disk storage means that the cache may not be able to accommodate all data, thus limiting the scale and effectiveness of the cache. The data in memory is non-persistent and will be lost once the system is powered off or restarted. This requires the cache system to have a data backup and recovery mechanism to ensure data integrity and consistency.

[0045] If there is data inconsistency between the cache and the database, business logic errors may occur. Cache update strategies, such as write penetration, write bypass, and read repair, need to be carefully designed to maintain data consistency. The cache update strategy needs to take into account the timeliness of the data, data consistency, and how to update the cache efficiently, which increases the complexity of the system. Cache penetration refers to the situation where the queried data does not exist in both the database and the cache, causing each request to fall on the database, increasing the burden on the database. Cache avalanche refers to the situation where a large amount of cache data becomes invalid at the same time, causing a large number of requests to flow into the database, which may cause the system to crash. In a multi-threaded or distributed environment, concurrent access to the same cache data may cause resource competition and lock contention, affecting system performance. Memory caches require regular management and maintenance, including monitoring, optimization, and fault recovery, which increases the operating cost of the system. Memory is usually more expensive than disk storage, so large-scale deployment of memory cache systems may result in higher hardware costs. A single-node memory cache system may have difficulty coping with large-scale data and high-concurrency requests, and a distributed cache solution is required to improve scalability and fault tolerance. The configuration and tuning of the cache system may be complex, requiring a deep understanding of business requirements and system architecture to achieve optimal performance and efficiency.

[0046] To this end, the present invention proposes a method for an efficient memory cache system, which can ensure a high cache hit rate, reduce cache pollution, ensure memory security and concurrency, and simplify the development and maintenance of the cache system.

[0047] Before describing the embodiments of the present invention, some terms or concepts involved in the embodiments of the present invention are explained.

[0048] LRU strategy: LRU (Least Recently Used) strategy is a commonly used cache eviction algorithm, which is used to decide which entry should be removed to make room for new data when the cache is full. The basic idea of ​​LRU strategy is that if the data has not been accessed in the recent period of time, then it is less likely to be accessed in the future, so this type of data can be considered "least recently used" and is a good candidate for cache eviction.

[0049] LFU strategy: LFU (Least Frequently Used) strategy is a cache eviction algorithm, which is mainly used to decide which entry should be removed when the cache is full. Unlike LRU (Least Recently Used) strategy, LFU strategy focuses on the access frequency of data rather than the access time. The core idea of ​​LFU algorithm is: if a piece of data has been accessed less frequently in the past, then it is less likely to be accessed in the future, so this type of data can be eliminated first.

[0050] FIFO strategy: FIFO (First In First Out) strategy is a simple cache elimination or data structure management algorithm, which follows the principle that the earliest data should be removed first. This strategy is used in various systems, including cache management in computer science, process scheduling in operating systems, transaction processing in database management, and data buffering in hardware design.

[0051] TTL strategy: TTL (Time To Live) strategy is a cache management and data elimination strategy, which is mainly used to deal with data entries with a limited life cycle. In the TTL strategy, each data entry is assigned an expiration time. When this time is reached, the data entry will be considered expired and removed from the cache. This strategy is particularly suitable for scenarios with high data validity requirements, such as dynamic pricing, real-time data analysis, short-term authentication tokens, etc.

[0052] Rust language: Rust is a system-level programming language that focuses on speed, memory safety, and concurrency. Rust is designed to solve common problems in C and C++, especially memory safety issues, without sacrificing performance. Its syntax is somewhat similar to C++, but it provides stricter memory management and ownership models, which allows programmers to write safer and more reliable code while enjoying runtime performance close to C or C++.

[0053] Compact structure: Compact refers to an optimized form of data structure or storage layout, which aims to reduce memory usage, improve data access efficiency, or simplify data processing.

[0054] Fast data structure: A data structure that is very efficient in specific operations and can quickly perform operations such as search, insertion, and deletion.

[0055] Exemplary embodiments of the present invention are described below with reference to the accompanying drawings.

[0056] Figure 1 A flow chart of a method for an efficient memory cache system according to an example embodiment is shown.

[0057] See also Figure 1 The present invention proposes a high-performance memory cache system that significantly reduces cache pollution while ensuring a high cache hit rate. In addition, the cache system provides flexible configuration options, and users can make trade-offs between time and space efficiency according to actual scenarios. The present invention is implemented in Rust language, which ensures memory safety and concurrency, and also greatly simplifies the development and maintenance of the cache system.

[0058] In S101 , a cache instance is created according to a specified cache capacity and an estimated number of objects.

[0059] The cache instance includes a small queue and a main queue. The small queue is used to store objects that are recently accessed or have a frequency of access lower than an access threshold, and the main queue is used to store objects that have a frequency of access equal to and / or greater than the access threshold.

[0060] When the program is initialized and started, a cache instance is created according to the configuration. Based on the specified cache capacity and the estimated number of objects, several key steps are usually involved, namely determining the cache strategy, selecting the appropriate cache data structure, and configuring cache parameters.

[0061] First, determine the cache strategy. Common strategies include but are not limited to LRU strategy, LFU strategy, FIFO strategy and TTL strategy. LRU strategy is the least recently used strategy. When the cache is full, the object that has not been used for the longest time is removed. LFU strategy is the least frequently used strategy. The object with the lowest access frequency is removed. FIFO strategy is a first-in-first-out strategy. The earliest added object is removed. TTL strategy is a time-based strategy. Each object has a survival time and is automatically removed when it times out.

[0062] According to the selected cache strategy, a suitable cache data structure is selected. The present invention combines FIFO and LFU. The LFU strategy can be implemented by using a hash table combined with a priority queue (minimum heap), and the FIFO strategy can be implemented by using a queue.

[0063] Configure cache parameters such as cache capacity and number of objects. Cache capacity (total_weight_limit) defines the maximum size of the cache, usually in bytes or number of objects. Object number presets the initial size of the cache and the choice of optimized data structure. Object size estimates the average size of each object to better manage cache space. Details of the replacement strategy, such as the count update rule of LFU, etc.

[0064] According to some embodiments, all data objects such as int (integer data type), string (text data type), byte (character data type), struct (structure) can be used as cache objects. It is up to the developer to use the cache system and call write operations, and to clarify the usage scenarios and advantages of the cache system, as well as to write objects to the cache in appropriate places and read them from the cache when the cache objects are needed elsewhere.

[0065] In S103, the object to be cached is written to the cache instance.

[0066] Write the cached object to the cache instance, and always write to the small queue. During the writing process, if the cache capacity is insufficient, some objects will be eliminated according to the FIFO strategy until there is enough space to accommodate new objects.

[0067] Identify the object to be written to the cache, and the unique key used to reference this object. The key should be unique enough so that the object can be accurately retrieved from the cache later. Call the write method to write the object to the cache using the write method provided by the memory cache system, passing the key and value as parameters, and write the cache object according to the cache policy.

[0068] According to some embodiments, a read method is called to query the corresponding value according to the key of the object to be cached, wherein if the key exists, the corresponding value is returned and the access frequency of the object to be cached is increased; if the key does not exist, a null value is returned. A get method is called to query the corresponding value according to the key, wherein if the key exists, the corresponding value is returned and the access frequency of the cached object is increased; if the key does not exist, a null value None is returned.

[0069] According to some embodiments, when reading the cached key-value object, the corresponding object metadata is updated, and the object metadata includes the access frequency and the queue to which it belongs. The metadata is used to guide cache eviction decisions and ensure efficient use of the cache. When reading a cached object, the cache system will update the object's metadata, such as the access frequency, the queue to which it belongs, etc. These metadata are used to guide cache eviction decisions and ensure efficient use of the cache.

[0070] When the number of access frequencies of the cached key-value object is greater than the access threshold, the cached key-value object is moved from the small queue to the main queue.

[0071] In S105, when the cache capacity is insufficient, cache key-value objects are preferentially eliminated from the small queue.

[0072] According to some embodiments, the cache key-value objects whose access frequency is lower than the frequency elimination threshold are preferentially eliminated from the small queue. If the small queue is empty, the cache key-value objects are eliminated from the main queue, and the cache key-value objects whose access frequency is lower than the frequency elimination threshold and / or whose residence time is lower than the residence time elimination threshold are eliminated.

[0073] According to some embodiments, when the cache capacity is insufficient, the cache system will automatically eliminate some objects to make room for new objects. The elimination process follows the FIFO strategy, and preferentially eliminates cache objects with low access frequency from the small queue; if the small queue is empty, cache objects with low access frequency and long residence time are eliminated from the main queue.

[0074] According to some embodiments, the elimination method of eliminating objects from the main queue when the small queue is empty is a method for handling complex cache scenarios for the following reasons: preventing deadlock, if the small queue is empty or all objects are promoted to the main queue, the system needs a method to continue the elimination operation; maintaining the total weight limit, even if the small queue has space, if the total weight exceeds the limit, the system still needs to eliminate objects; adapting to dynamic changes, in a concurrent environment, the queue state may change between inspection and execution; optimizing cache content, sometimes eliminating less used objects in the main queue may be more beneficial than adding new objects to the small queue.

[0075] The system uses several mechanisms to distinguish and handle objects of different frequencies and residence times. The TinyLFU estimator is used to track the access frequency of objects, and the uses counter is used to track the number of times an object is accessed.

[0076] The eliminated object will inform the method caller through the return value. The memory cache system dynamically adjusts the cache write and elimination strategy. When the workload of the memory cache system changes, the cache adaptively adjusts the capacity of the small queue and the main queue, as well as the residence time and access frequency requirements of the cache key value objects. The cache system dynamically adjusts the cache admission and elimination strategy through a combination of LFU and FIFO strategies. When the workload changes, the cache will adaptively adjust the small queue and the main queue, as well as the residence time and access frequency requirements of the object.

[0077] Figure 2 A schematic diagram of a working process of a memory cache system according to an example embodiment is shown.

[0078] The existing cache system has high space complexity. The traditional LFU algorithm needs to maintain the access frequency counter of each object, which has high space complexity. As the number of cached objects increases, the memory occupied by the frequency counter will increase sharply, resulting in excessive system memory pressure. The traditional LFU algorithm eliminates objects based on the lowest access frequency. In some scenarios, this strategy may cause cache pollution problems. For example, for scanning access patterns, LFU will keep a large number of one-time access objects in the cache for a long time, squeezing the space of hot objects and reducing the overall cache hit rate.

[0079] The present invention provides a high-performance cache system, which aims to improve the cache hit rate and resource utilization efficiency in a multi-threaded environment. The cache system of the present invention combines multiple data structures and algorithms, and through flexible cache management strategies and optimized concurrent processing mechanisms, it minimizes memory overhead and cache replacement loss while ensuring high performance.

[0080] The cache system of the present invention adopts a multi-shard skip list design (Sharded Skip List Design). N-shard skip list means dividing a large skip list into N smaller skip lists (shards), and each shard is responsible for storing a portion of the data. This design has high memory efficiency, and the skip list itself saves more memory than the hash table, and is more flexible after sharding. The search complexity of the skip list is O(log n), but due to sharding, it is actually O(log(n / N)). When N is large, it is close to constant time, and an average constant time search can be achieved. Different shards can be accessed concurrently, reducing lock contention and having good concurrency performance.

[0081] Compact structure is used to achieve efficient memory cache management. Compact uses SkipMap (a mapping structure based on skip list) across multiple shards. Each shard contains a SkipMap instance, thus providing constant time complexity search and update operations in scenarios with tight memory usage. This design makes full use of the orderliness of the skip list and the parallelism of the shards, improving the cache search speed and insertion and deletion efficiency.

[0082] The cache system of the present invention provides a Fast data structure, which implements high-speed cache access through concurrent HashMap. Fast is suitable for scenarios that require high throughput, provides fast read and write operations, and supports multi-threaded concurrent access. By flexibly selecting Compact or Fast data structures, the cache system can achieve optimal performance and memory usage in different application scenarios. For example, the storage backend, Fast <t>Based on concurrent hash map, Compact <t>Based on N-shard skip list; Fast uses more memory but provides faster access speed, while Compact uses less memory but has a slight sacrifice in access speed; Fast provides the best query performance and is suitable for high-frequency access scenarios, while Compact provides better memory efficiency at the expense of some performance, and is suitable for large-scale cache items or memory-constrained environments.

[0083] The cache system records the access frequency through multiple hash functions and counter matrices, and performs aging processing on the frequency data within a cycle to improve the accuracy of frequency estimation. The present invention proposes a data structure and algorithm called "Count-Min Write" for efficiently estimating the frequency of occurrence of elements in a data stream and recording the access frequency. This method has the advantages of high space efficiency, fast query speed, and parallelization, and is particularly suitable for large-scale data stream processing.

[0084] The functions of frequency estimation include: tracking object access patterns, the frequency estimator records the frequency of each object being accessed (or attempted to be written). This helps identify hot data (frequently accessed data); assisting cache replacement decisions, when the cache is full, frequency information is used to decide whether the existing cache items should be replaced; implementing the LFU strategy, the Least Frequently Used strategy, which is an improved cache replacement algorithm that uses frequency information to make smarter cache decisions.

[0085] Frequency estimation plays a key role in cache systems. It does not directly trigger write operations, but optimizes cache performance by influencing write decisions. It enables the system to make smarter cache replacement decisions, improve cache efficiency, and adapt to various data access patterns. This mechanism enables the cache system to consider not only the recent use of data, but also its long-term access frequency, thereby achieving more efficient cache management.

[0086] An embodiment of an algorithm implementation is described below.

[0087] The data structure of Count-Min Write is a two-dimensional table with a small number of rows, usually 4-5 rows, and a large number of columns, such as hundreds or thousands of columns. Each cell is a counter with an initial value of 0. Prepare independent hash functions equal to the number of rows. These functions can map input elements to column indices. When an element arrives, use each hash function to calculate the column index, and then add 1 to the column counter of the corresponding row, which is an insertion operation. The query operation estimates the frequency of an element, and also uses all hash functions to calculate the column index, and then takes the minimum value of all corresponding counters.

[0088] Due to possible hash collisions, some counters may be shared by multiple different elements, resulting in higher counts, but taking the minimum value can minimize this effect, because it is unlikely that all rows will have a serious collision.

[0089] The advantages of Count-Min Write are that it is space efficient and does not need to allocate storage space for each possible element, making it suitable for processing huge data sets. In terms of time efficiency, the time complexity of insertion and query operations is constant. It can be parallelized, and operations on different rows can be performed in parallel to increase processing speed. By adjusting the number of rows and columns, a balance can be achieved between accuracy and resource consumption, which is adjustable. The Count-Min Write algorithm provides an innovative method for quickly estimating the frequency of elements in large-scale data streams with limited resources. While maintaining high efficiency, it provides sufficiently accurate frequency estimates for many practical applications, bringing new possibilities for big data processing and analysis.

[0090] The following example describes an aging process.

[0091] Decay trigger mechanism:

[0092] The decay process is triggered in the incr method of TinyLfu:

[0093] pub fn incr<T: Hash> (&self, key: T) ->u8 {

[0094] let window_size = self.window_counter.fetch_add(1, Ordering::Relaxed);

[0095] if window_size == self.window_limit || window_size>self.window_limit* 2 {

[0096] self.window_counter.store(0, Ordering::Relaxed);

[0097] self.estimator.age(1); / / right shift 1 bit

[0098] }

[0099] self.estimator.incr(key)

[0100] }

[0101] Each time incr is called, window_counter is increased by 1; window_counter is checked to see if it has reached window_limit, and if so, the decay process is triggered; window_counter is reset to 0.

[0102] The attenuation is implemented as follows:

[0103] The decay operation is implemented in the age method of Estimator:

[0104] pub fn age(&self, shift: u8) {

[0105] for (slot, _) in self.estimator.iter() {

[0106] for counter in slot.iter() {

[0107] let c = counter.load(Ordering::Relaxed);

[0108] counter.store(c>>shift, Ordering::Relaxed);

[0109] }

[0110] }

[0111] }

[0112] Traverse all counters and perform a right shift operation on each counter (here it is a 1-bit right shift). The right shift operation halves the count value, which is the attenuation effect.

[0113] The significance of decay lies in time decay, where recent accesses are more important than old accesses; preventing overflows, where long-term operations cause the counter to reach its maximum value; and adapting to changes, where the cache is allowed to adapt to changes in data access patterns.

[0114] This patented invention cleverly implements an adaptive frequency estimation system. While maintaining estimation accuracy, it also ensures that the system can adapt to changing data access patterns. This mechanism strikes a good balance between memory efficiency, computational efficiency, and accuracy, allowing the cache system to perform well in a variety of complex cache scenarios.

[0115] Figure 3 A schematic diagram showing a process of removing a queue object according to an example embodiment.

[0116] The cache system is designed with an efficient hierarchical cache queue structure FiFoQueues, which includes small queues and main queues. The small queue mainly stores recent or less frequently accessed cache objects. When the access frequency of these objects increases, they will be promoted to the main queue to reduce the probability of being replaced. The main queue is used to store objects with higher frequency, and the cache hit rate and management efficiency are improved through hierarchical management.

[0117] FiFoQueues is the core component for implementing the eviction strategy in this cache system. It is an improved FIFO (First-In-First-Out) cache replacement strategy that aims to improve the cache hit rate and performance. The structure is as follows:

[0118] struct FiFoQueues <t>{

[0119] total_weight_limit: usize,

[0120] small: SegQueue <key>,

[0121] small_weight: AtomicUsize,

[0122] main: SegQueue <key>,

[0123] main_weight: AtomicUsize,

[0124] estimator: TinyLfu,

[0125] _t: PhantomData <t>,

[0126] }

[0127] In the dual-queue system, the small queue is used to store newly entered items in the cache, and the main queue is used to store popular items upgraded from the small queue. estimator: TinyLFU is a frequency estimator used to decide whether an item should be accepted or upgraded.

[0128] The advantages of the FiFoQueues strategy are: fast admission, new items quickly enter the small queue to adapt to sudden access patterns; protection of hot spots, frequently accessed items are upgraded to the main queue and are not easily evicted; efficient cleanup, infrequently used items are quickly removed from the small queue; anti-scanning, can effectively deal with a large number of one-time access situations.

[0129] FiFoQueues implements an efficient dual-queue system that combines the simplicity of FIFO and the frequency awareness of LFU (Least Frequently Used). Through the collaboration of small queues and main queues, and the assistance of estimators, it can provide good cache performance in various access modes, balancing the cache response speed and hit rate.

[0130] The cache system uses a frequency promotion strategy to manage the life cycle of cached objects. When the access frequency of a cached object increases, it will be promoted from the small queue to the main queue to avoid accidental deletion of frequently accessed objects. When replacing the cache, an adaptive strategy is adopted to eliminate low-frequency objects from the small queue first. If the demand cannot be met, the object will be eliminated from the main queue; this strategy achieves a higher cache hit rate and more effective cache management.

[0131] The cache system provides flexible cache structure configuration capabilities. Users can choose the most appropriate cache implementation (such as Compact or Fast) according to specific application scenarios. If the application is not sensitive to memory usage and requires the highest performance, choose Fast. If the application needs to process a large number of cache items or runs in a memory-constrained environment, such as an embedded system or a high-concurrency server, choose Compact. This flexible design can achieve the best balance between performance and memory usage under different resource environments and application requirements.

[0132] Figure 4 A block diagram of a computing device is shown according to an exemplary embodiment.

[0133] like Figure 4 As shown, computing device 30 includes processor 12 and memory 14. Computing device 30 may also include bus 22, network interface 16, and I / O interface 18. Processor 12, memory 14, network interface 16, and I / O interface 18 may communicate with each other via bus 22.

[0134] The processor 12 may include one or more general-purpose CPUs (Central Processing Units, processors), microprocessors, or application-specific integrated circuits, etc., for executing relevant program instructions. According to some embodiments, the computing device 30 may also include a high-performance graphics card (GPU) 20 for accelerating the processor 12.

[0135] The memory 14 may include a machine system readable medium in the form of a volatile memory, such as a random access memory (RAM), a read-only memory (ROM) and / or a cache memory. The memory 14 is used to store one or more programs including instructions and data. The processor 12 may read the instructions stored in the memory 14 to execute the above-mentioned method according to the embodiment of the present invention.

[0136] The computing device 30 may also communicate with one or more networks via the network interface 16. The network interface 16 may be a wireless network interface.

[0137] The bus 22 may include an address bus, a data bus, a control bus, etc. The bus 22 provides a path for exchanging information between components.

[0138] It should be noted that, in the specific implementation process, the computing device 30 may also include other components necessary for normal operation. In addition, those skilled in the art may understand that the above device may only include components necessary for implementing the embodiments of this specification, and need not include all components shown in the figure.

[0139] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when the program is executed by a processor. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), a network storage device, a cloud storage device, or any type of medium or device suitable for storing instructions and / or data.

[0140] An embodiment of the present invention further provides a computer program product, which includes a computer program. The computer program is operable to enable a computer to execute part or all of the steps of any one of the methods described in the above method embodiments.

[0141] Those skilled in the art can clearly understand that the technical solution of the present invention can be implemented with the help of software and / or hardware. "Unit" and "module" in this specification refer to software and / or hardware that can independently complete or cooperate with other components to complete specific functions, where the hardware can be, for example, a field programmable gate array, an integrated circuit, etc.

[0142] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0143] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0144] In the several embodiments provided by the present invention, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0145] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0146] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0147] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention.

[0148] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0149] The exemplary embodiments of the present invention are specifically shown and described above. It should be understood that the present invention is not limited to the detailed structures, configurations or implementations described herein; on the contrary, the present invention is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended clauses.< / t> < / key> < / key> < / t> < / t> < / t>

Claims

1. A method for a memory cache system, wherein the memory cache system is used to store key-value objects, characterized in that: The method comprises: Create a cache instance according to the specified cache capacity and the estimated number of objects, the cache instance including a small queue and a main queue, the small queue is used to store objects that are recently accessed or have a frequency of access lower than an access threshold, and the main queue is used to store objects that have a frequency of access equal to and / or greater than the access threshold; Write the cached object to the cache instance, and always write to the small queue; When the cache capacity is insufficient, cache key-value objects are preferentially eliminated from the small queue; If the small queue is empty, the cache key-value object is eliminated from the main queue.

2. The method according to claim 1, characterized in that Writing the cached object to the cache instance includes: Call the read method to query the corresponding value according to the key of the object to be cached, wherein if the key exists, the corresponding value is returned and the access frequency of the object to be cached is increased; if the key does not exist, a null value is returned.

3. The method according to claim 1, characterized in that Also includes: When the number of access frequencies of the cache key-value object is greater than the access threshold, the cache key-value object is moved from the small queue to the main queue.

4. The method according to claim 1, characterized in that: When the cache capacity is insufficient, cache key-value objects are preferentially eliminated from the small queue, including: The cache key-value objects whose access frequency is lower than the frequency elimination threshold are preferentially eliminated from the small queue.

5. The method according to claim 1, characterized in that Eliminating the cache key-value object from the main queue includes: When the small queue is empty, the cache key-value objects whose access frequency is lower than a frequency elimination threshold and / or whose residence time is lower than a residence time elimination threshold are eliminated.

6. The method according to claim 1, characterized in that Also includes: When reading the cache key-value object, the corresponding object metadata is updated, and the object metadata includes the access frequency and the queue to which it belongs.

7. The method according to claim 1, characterized in that Also includes: Dynamically adjust the write and eviction strategies of the memory cache system; When the workload of the memory cache system changes, the capacity of the small queue and the main queue, and / or the residence time and / or access frequency requirements of the cache key-value objects are adaptively adjusted.

8. A computer program product, characterized in that The method comprises a computer program, which implements the method according to any one of claims 1 to 7 when being executed by a processor.

9. A computing device, characterized in that include: processor; as well as A memory storing a computer program, wherein when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is implemented.

Images