Managing least recently used data cache with persistent principals

Abstract

A method for managing a data cache includes storing a cache management list that includes a plurality of entries and has a tail portion stored in a first storage device and recording recently accessed data items stored in the data cache, a body portion stored in a second storage device and recording less recently accessed data items stored in the data cache, and a head portion stored in the first storage device and recording least recently accessed data items stored in the data cache, and at each iteration of a plurality of iterations: receiving at least one data access request; recording the data access request in the tail portion; identifying a plurality of duplicate entries that exist in the body and tail; and removing each of the plurality of duplicate entries from the body in the second storage device according to a physical organization of the plurality of duplicate entries in the second storage device.

Description

Technical Field

[0001] Some embodiments described in this disclosure relate to managing data caches, and more specifically, but not exclusively, to using fast access memory to manage data caches. Background Technology

[0002] In computing, a data cache is a component that temporarily stores data so that future requests for that data can be served faster than the first request. A data cache can be hardware, software, or both. The data stored in the data cache can be the result of an earlier computation or a copy of data stored elsewhere.

[0003] For the sake of brevity, the term "cache" will henceforth be used to refer to "data cache," and these terms are used interchangeably.

[0004] A cache client is an entity that accesses data stored in a cache, such as a processing unit or application. In some systems, the cache is local to the cache client and separate from the bulk storage device that stores the data, allowing the cache client to access a copy of the data stored in the cache much faster than accessing the data on bulk storage. For example, when data is stored on a hard disk drive (HDD), some of the data can be stored in a cache residing in random access memory (RAM). Access to RAM is significantly faster than access to an HDD, so data stored in a cache in RAM can be processed much faster than other data stored on an HDD. Another example is implementing a cache using fast memory, which is more expensive than other memory with slower access times for storing data. Yet another example is when data is stored on network-connected memory. The cache client can store a copy of some data on local storage, which can be volatile or non-volatile. In this example, accessing data on local storage does not require communication over the network and is therefore faster.

[0005] A cache may not store a copy of all the data stored on a mass storage device. Some caches are used to store data that cache clients have recently accessed or frequently accessed, thereby improving cache client performance by processing such requests faster than when processing requests from mass storage. When a cache only stores a copy of a portion of the data stored on mass storage, it is necessary to determine whether a data access request from a cache client can be processed from the cache, or whether access to the mass storage device is required. The term cache metadata refers to information documented in the cache about multiple data items, for example, a copy of each data item is stored on mass storage and used to manage the cache. Such information may include when a copy of a data item was accessed and the source of the data item on permanent storage, such as a storage address or filename.

[0006] Over time, cache clients may frequently access new data items that haven't been accessed before, and may need to add copies of these new data items to the cache. When cache capacity is limited, it may be necessary to remove a copy of another data item from the cache to create a new copy. The least recently used (LRU) cache is the cache that removes the least recently used data items first when data items need to be removed from the cache. Summary of the Invention

[0007] According to one aspect of the present invention, a method, computer program product, and / or system for managing a data cache is provided, which performs the following operations (not necessarily in the following order): (i) storing a cache management list comprising a plurality of entries, the cache management list comprising: (a) a tail portion stored in a first storage device and recording a plurality of recently accessed data items stored in the data cache, and (b) a body portion stored in a second storage device and recording a plurality of less recently accessed data items stored in the data cache; (ii) in each cache management iteration of a plurality of cache management iterations: (a) receiving a first data access request; (b) recording the first data access request in the tail portion; (c) identifying a plurality of duplicate entries associated with the first data access request present in the body portion and the tail portion; and (d) removing each of the plurality of duplicate entries from the body portion according to the physical organization of the plurality of duplicate entries in the second storage device. Attached Figure Description

[0008] Some embodiments are described herein by way of example only with reference to the accompanying drawings. Reference will now be made specifically to the drawings, with emphasis that the details shown are by way of example and for the purpose of illustrative discussion of the embodiments. In this regard, the description in conjunction with the drawings will make it clear to those skilled in the art how to practice the embodiments.

[0009] Figure 1 This is a block diagram of a cache management list according to at least one embodiment of the present invention;

[0010] Figure 2 This is a schematic block diagram of a system according to at least one embodiment of the present invention;

[0011] Figure 3 This is a flowchart schematically illustrating an optional process for managing a cache according to at least one embodiment of the present invention;

[0012] Figure 4 This is a flowchart schematically illustrating an optional process for cache miss operations according to at least one embodiment of the present invention;

[0013] Figure 5 This is a flowchart schematically illustrating an optional process for managing access to multiple data items; and

[0014] Figure 6 This is a block diagram of possible hardware and software environments for software and / or methods according to at least one embodiment of the present invention. Detailed Implementation

[0015] The following description focuses on using a cache management list that includes multiple entries, and more specifically on managing the least recently used (LRU) cache. However, the following description can be applied to other cache replacement strategies besides LRU, where another test can be applied to differentiate between multiple parts of the cache management list, such as most recently used, first-in-first-out, and last-in-last-out.

[0016] A cache hit occurs when a cache client accesses a data item that exists in the cache. A cache miss occurs when a cache client requests access to a data item that does not exist in the cache (the missed data item). When a cache miss occurs, it is necessary to access mass storage to retrieve the missed data item. Additionally, it may be necessary to add a copy of the missed data item to the cache.

[0017] Some embodiments of this disclosure describe systems and methods for managing memory caches, wherein the amount of metadata recording multiple data items in the cache exceeds the amount available in a first storage device for metadata. In such embodiments, the metadata is partitioned between a first storage device and a second storage device, wherein multiple recently accessed data items are recorded in the first storage device and multiple less recently accessed data items are recorded in the second storage device. Optionally, in each of multiple cache management iterations, the metadata in the first and second storage devices is synchronized.

[0018] The foregoing and other objectives are achieved through the features of the independent claims. Further implementations will be apparent from the dependent claims, the specification, and the drawings.

[0019] According to a first aspect of the invention, a method for managing a data cache includes storing a cache management list comprising multiple entries and having: a tail portion stored in a first storage device and recording multiple recently accessed data items stored in the data cache; and a body portion stored in a second storage device and recording multiple less recently accessed data items stored in the data cache. Optionally, the method includes, in each of multiple cache management iterations, receiving at least one data access request, recording at least one data access request in the tail portion, identifying multiple duplicate entries present in the body portion and the tail portion, and removing each of the multiple duplicate entries from the body portion in the second storage device according to the physical organization of the multiple duplicate entries in the second storage device. Recording data accesses in the tail portion while the tail portion resides in the first storage device helps increase the throughput of the managed cache compared to when the first storage device is not large enough to store the entire cache management list and the entire cache management list resides on the second storage device, for example, when accesses to the first storage device are faster than accesses to the second storage device. Compared to removing multiple duplicate entries in a random order, removing each of the multiple duplicate entries from the main body based on the physical organization of the multiple duplicate entries in the second storage device reduces the amount of time required to update the main body.

[0020] According to a second aspect of the invention, a system for managing a data cache includes at least one hardware processor adapted to: store a cache management list comprising a plurality of entries and having: a tail portion stored in a first storage device connected to the at least one hardware processor and recording a plurality of recently accessed data items stored in the data cache; and a body portion stored in a second storage device connected to the at least one hardware processor and recording a plurality of less recently accessed data items stored in the data cache; and in each of a plurality of cache management iterations: receiving at least one data access request; recording at least one data access request in the tail portion; identifying a plurality of duplicate entries present in the body portion and the tail portion; and removing each of the plurality of duplicate entries from the body portion in the second storage device according to the physical organization of the plurality of duplicate entries in the second storage device.

[0021] According to a third aspect of the invention, a system for managing access to a plurality of data items includes: a data cache storing at least some of the plurality of data items; and at least one hardware processor connected to the data cache and adapted to manage the data cache by: storing a cache management list comprising a plurality of entries and having: a tail portion stored in a first storage device and recording a plurality of recently accessed data items stored in the data cache; and a body portion stored in a second storage device and recording a plurality of less recently accessed data items stored in the data cache; and in each of a plurality of cache management iterations: receiving at least one data access request; recording at least one data access request in the tail portion; identifying a plurality of duplicate entries present in the body portion and the tail portion; and removing each of the plurality of duplicate entries from the body portion in the second storage device according to the physical organization of the plurality of duplicate entries in the second storage device. Managing the data cache as described above helps to increase the throughput of the system for managing access to a plurality of data items.

[0022] In implementations of the first and second aspects, the cache management list further includes a header portion stored in the first storage device and records a plurality of least recently accessed data items stored in the data cache. Optionally, the method further includes, in at least some of multiple cache management iterations: identifying new plurality of least recently accessed entries in the body portion, recording the new plurality of least recently accessed data items stored in the data cache; and copying the new plurality of least recently accessed entries from the body portion in the second storage device to the header portion in the first storage device. Compared to storing the entire cache management list on the second storage device when the first storage device is not large enough to store the entire cache management list, storing the header portion in the first storage device helps increase the throughput of the managed cache, for example, when accesses to the first storage device are faster than accesses to the second storage device.

[0023] In another implementation of the first and second aspects, the method further includes copying the tail portion to a second storage device. Optionally, the body portion is stored as multiple files in the second storage device. Optionally, copying the tail portion to the second storage device includes adding the file including the tail portion to the multiple files. When the second storage device is a non-volatile storage device, copying the tail portion to the second storage device helps to make the tail portion persistent. For example, adding files helps to reduce the amount of time required to copy the tail portion to the second storage device compared to updating existing files.

[0024] In an additional implementation of the first and second aspects, recording at least one data access request in the tail portion includes at least one of the following: updating an entry in the tail portion, recording another access request for the corresponding data item, wherein the corresponding data item is recorded by the other access request, and moving the updated entry to the end of the tail portion; and adding a new entry to the tail portion. Optionally, if the tail portion is identified as full, at least one other cache management iteration in a plurality of cache management iterations is performed. Optionally, at least some of the plurality of cache management iterations are performed periodically. If the tail portion is identified as full, performing at least one other cache management iteration helps to identify candidates for eviction from the head portion rather than from the tail portion, thereby increasing cache management throughput by reducing the likelihood of future cache misses. Furthermore, periodically performing at least some of the plurality of cache management iterations helps to increase cache management throughput by increasing the identification of candidates for eviction from the head portion and reducing the likelihood of performing cache management iterations in response to data access requests.

[0025] In an additional implementation of the first and second aspects, the method further includes, in at least one cache management iteration among multiple cache management iterations, identifying a data access request in at least one data access request that requests access to a data item not stored in the data cache, and inserting the data item into the data cache. Optionally, the cache management list further has a header portion stored in a first storage device and records a plurality of least recently accessed data items stored in the data cache. Optionally, the method further includes, in at least one cache management iteration, identifying at least one entry in the header portion that is not in the tail portion, recording access to at least one data item stored in the data cache, and removing at least one entry from the header portion. Optionally, if at least one entry in the header portion that is not in the tail portion is not identified, at least one additional cache management iteration among multiple cache management iterations is performed. Performing at least one additional cache management iteration in multiple cache management iterations when at least one entry in the head portion that is not in the tail portion is not identified helps identify candidates to be evicted from the cache from the head portion rather than from the tail portion, thereby increasing cache management throughput by reducing the likelihood of future cache misses.

[0026] In another implementation of the first and second aspects, the ratio between the first access time of the first storage device and the second access time of the second storage device is less than an access time ratio threshold. Using a first storage device with access times such that the ratio between the first access time of the first storage device and the second access time of the second storage device is less than the access time ratio threshold reduces the amount of time required to respond to memory access requests compared to when the entire cache management list is stored on the second storage device.

[0027] In another implementation of the first and second aspects, the first storage device is selected from a first set of digital memories, which includes: random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), hard disk drive, and solid-state drive; and the second storage device is selected from a second set of digital memories, which includes: DRAM, hard disk drive, solid-state drive, electrically erasable programmable read-only memory (EEPROM), NAND flash memory, network-connected storage device, and network storage device.

[0028] In another implementation of the first and second aspects, at least one of at least one data access request is received from a software object executed by at least one hardware processor. Optionally, at least one other data access request is received from at least one other hardware processor connected to the at least one hardware processor. Optionally, the at least one other hardware processor is connected to the at least one hardware processor via at least one digital communication network interface connected to the at least one hardware processor.

[0029] In the third aspect of the implementation, at least one hardware processor is further adapted to: receive at least one other data access request requesting access to at least one of a plurality of data items; identify at least one of at least some data items stored in a data cache; and, in response to the at least one data access request, access at least one data item in the data cache. Accessing data items in the data cache increases the system throughput for accessing multiple data items by reducing the amount of time data items spend on another storage device, for example, when the storage device is remote from at least one hardware processor.

[0030] Other systems, methods, features, and advantages of this disclosure will be or will become apparent to those skilled in the art after studying the following figures and detailed description. All such additional systems, methods, features, and advantages are intended to be included in this specification, within the scope of this disclosure, and protected by the appended claims.

[0031] Unless otherwise defined, all technical and / or scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. While similar or equivalent methods and materials may be used to implement or test embodiments, exemplary methods and / or materials are described below. In case of conflict, the patent specification (including definitions) shall prevail. Furthermore, materials, methods, and embodiments are illustrative only and not restrictive.

[0032] One possible way to manage a cache is by managing a cache management list containing multiple entries, each entry recording a data item stored in the cache. Each entry can include the most recent record of the corresponding data item being accessed by one or more data clients using the data cache. The cache management list can be sorted and has a beginning and an end, such that new entries are added to the end of the cache management list, a portion also known as the tail portion. In this cache management list, another portion at the beginning, also known as the head portion, includes the oldest of the multiple entries. See now. Figure 1The diagram illustrates a block diagram of an exemplary cache management list 100. In this example, the cache management list 100 includes a header portion 101, a body portion 102, and a tail portion 103. In this example, the header portion 101 includes at least one entry 111 that records an access to a corresponding data item associated with it at a first time, represented by t1. Additionally, in this example, the body portion 102 includes at least one other entry 121 that records another access to another corresponding data item associated with it at a second time, represented by t2. Additionally, in this example, the tail portion 103 includes at least one additional other entry 131 that records additional other accesses to additional corresponding data items associated with it at a third time, represented by t3.

[0033] As described above, in an LRU cache, when a data item needs to be removed from the cache, the least recently used data item is removed first. An LRU cache can be managed using a cache management list, where the head portion 101 includes one or more entries recording one or more least recently accessed data items, and the tail portion 103 contains one or more other entries recording one or more recently accessed data items. For example, in cache management list 100, t3 can be equal to or more recent than t1 and t2, meaning t3 is equal to or later than both t1 and t2. Similarly, in this example, t1 can be equal to or not more recent than t2 and t3, meaning t3 is equal to or earlier than both t2 and t3. One or more recently accessed data items can be one or more most recently accessed data items, meaning the time associated with any entry in the tail portion 103 can be no later than any other time associated with any other entry in the body portion 102 or the head portion 101. Multiple less recently accessed data items can be recorded by multiple other entries in the body portion 102 of the list. For example, t2 can be equal to or not more recent than t3, that is, t2 is equal to or earlier than t3.

[0034] According to some cache replacement strategies, when it is necessary to replace data items in the cache—that is, to remove one or more existing data items to make room for another data item—one or more entries, such as entry 131, are selected from the head portion 103, and the corresponding data item associated with the selected entry is replaced in the cache. In such a strategy, when a new data item is added to the cache, an entry recording access to the new data item is added to the tail portion 101.

[0035] To increase the throughput of cache clients accessing the cache, cache management lists are typically stored in memory or storage devices that are quickly accessed by the cache clients. For example, when the cache client is a hardware processor, the cache management list may be stored in a local memory component electrically coupled to the hardware processor. However, depending on the number of multiple data items with copies stored in the cache, and additionally or alternatively, depending on the amount of data describing access to each data item in the cache, the size of the cache management list may exceed the amount of storage allocated to store the cache management list. In the example above, the size of the cache management list may exceed the amount of local memory allocated to store the cache management list.

[0036] Because fast memory and high-speed memory are generally more expensive than slower storage devices or memory, increasing the amount of storage devices or memory used to store cache management lists may be limited by the cost of such devices or memory. One possible solution is to store the cache management list in a larger storage device, such as a hard disk drive. However, large storage devices are generally slower than other storage devices typically used to store cache management lists. Furthermore, in some technologies used for large storage devices, such as hard disk drives, the access time to an entry in the cache management list is affected by the physical organization of multiple entries in the cache management list within the storage device. Accessing more than one entry among multiple entries without considering their physical organization within the storage device can be much slower than accessing more than one entry based on their physical organization within the storage device. For example, when the cache management list is stored as a file on a hard disk drive, opening and closing the file to access one entry at a time can be much slower than opening the file and accessing all or one or more entries in the file before closing it.

[0037] Some embodiments described herein propose dividing a cache management list between two storage devices, wherein a tail portion residing in the first storage device records a plurality of recently accessed data items stored in the cache, and a body portion residing in the second storage device records a plurality of less recently accessed data items stored in the cache. The first storage device may be faster than the second storage device, such that a first access time, indicating the amount of time required to retrieve data from the first storage device, may be less than a second access time, indicating the amount of time required to retrieve data from the second storage device. The first storage device may be significantly faster than the second storage device, such that the ratio between the first access time and the second access time is less than an access time ratio threshold, for example, less than 0.1 or less than 0.01, i.e., the first storage device is at least 10 times or 100 times faster than the second storage device, respectively. For example, the first storage device may be static RAM (SRAM), and the second storage device may be RAM. In another example, the first storage device is RAM, and the second storage device is an HDD or solid-state drive (SSD). In yet another example, the first storage device is an SSD having a faster access time than another SSD used for the second storage device.

[0038] In such an embodiment, one or more data access requests are recorded in the tail portion. When the accessed data item is in the cache (cache hit), the access is recorded by adding an entry to the tail portion or updating an existing entry (e.g., by moving to the end of the tail portion). When a data item is added to the cache, an entry recording the access to that data item is added to the tail portion. When an entry and another entry both record accesses to the same data item, the entry is considered a duplicate of the other entry. This entry and other entries can record separate accesses to the same data item. Recording data accesses in the tail portion while the tail portion resides on the first storage device helps increase the throughput of the managed cache compared to when the first storage device is not large enough to store the entire cache management list and the entire cache management list resides on the second storage device, for example, when accesses to the first storage device are faster than accesses to the second storage device.

[0039] Because data access is recorded only in the tail portion, the tail can, over time, include one or more duplicate entries that are one or more entries in the body portion. In some embodiments described herein, this disclosure proposes identifying multiple duplicate entries present in both the body and tail portions and removing each of the multiple duplicate entries from the body portion in a second storage device. Optionally, removing each of the multiple duplicate entries from the body portion is based on the physical organization of the multiple duplicate entries in the second storage device. For example, when the multiple duplicate entries are located in multiple files in the second storage device, removing each of the multiple duplicate entries may include opening each of the multiple files once and removing all the multiple duplicate entries in the files before closing them. Removing each of the multiple duplicate entries from the body portion based on the physical organization of the multiple duplicate entries in the second storage device reduces the amount of time required to update the body portion compared to removing the multiple duplicate entries in a random order.

[0040] Optionally, identifying and removing multiple duplicate entries from the main portion is performed in each of multiple cache management iterations (sometimes referred to herein as management iterations). Optionally, one or more of the multiple management iterations are performed periodically. Optionally, for example, when a new entry needs to be added to the tail portion or after a new entry has been added to the tail portion, or if the tail portion is identified as full, one or more other management iterations are performed. Optionally, the tail portion is copied to a second storage device, for example, when the second storage device is a non-volatile memory, such as an HDD or SSD, and the first storage device is volatile, such as RAM.

[0041] Additionally, in some embodiments, the header portion of the cache management list is stored in a first storage device. Optionally, a copy of the header is stored in the first storage device. Optionally, when a data item needs to be removed from the cache, one or more entries in the header portion that are not in the tail portion are identified, wherein one or more entries record access to the data item. Optionally, when a data item is removed from the cache, one or more entries are removed from the header portion. Compared to storing the entire cache management list on a second storage device when the first storage device is not large enough to store the entire cache management list, for example, when access to the first storage device is faster than access to the second storage device, storing the header portion in the first storage device helps increase the throughput of the managed cache.

[0042] Optionally, if the header portion is identified as empty, one or more additional management iterations may be performed within a larger set of management iterations.

[0043] Before explaining at least one embodiment in detail, it should be understood that the embodiments are not necessarily limited in their application to the details of the construction and arrangement of the components and / or methods set forth in the following description and / or shown in the drawings and / or examples. The implementations described herein can be other embodiments or can be practiced or performed in various ways.

[0044] The embodiments may be systems, methods, and / or computer program products. A computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to perform aspects of the embodiments.

[0045] Computer-readable storage media can be tangible devices capable of retaining and storing instructions used by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0046] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a suitable computing / processing device, or downloaded via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network) to an external computer or external storage device. The network may include copper cables, optical fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the suitable computing / processing device.

[0047] Computer-readable program instructions used to perform the operations of the embodiments may be assembly instructions, instruction set architecture (ISA) instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, a field-programmable gate array (FPGA), or a programmable logic array (PLA) may execute the computer-readable program instructions to personalize the electronic circuitry in order to perform aspects of the embodiments by utilizing state information from the computer-readable program instructions.

[0048] This document describes aspects of embodiments with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments. It will be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0049] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and / or other devices to operate in a particular manner, such that the computer-readable storage medium in which the instructions are stored includes an article of writing comprising instructions for implementing aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0050] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer-implemented process, such that the instructions, which execute on the computer, other programmable apparatus or other device, perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0051] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing a specified logical function(s). In some alternative implementations, the functions mentioned in the blocks may occur in a non-linear order as shown in the figures. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order, depending on the functions involved. It will also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

[0052] Now for reference Figure 2 This diagram illustrates a schematic block diagram of an exemplary system 200 according to some embodiments. In such embodiments, at least one hardware processor 211 is connected to a first storage device 201 and a second storage device 202. Optionally, the first storage device 201 is selected from a first group of storage devices, including RAM, SRAM, dynamic random access memory (DRAM), HDD, and SSD. Optionally, the second storage device 202 is selected from a second group of storage devices, including DRAM, HDD, SSD, electrically erasable programmable read-only memory (EEPROM), NAND flash memory, network-connected storage devices, and network storage devices. Optionally, the first storage device 201 is faster than the second storage device 202, such that a first access time of the first storage device 201 is less than a second access time of the second storage device 202. Optionally, the ratio between the first access time and the second access time is less than an access time ratio threshold. Some examples of access time ratio thresholds are 0.1, 0.5, 0.01, 0.035, and 0.0001.

[0053] For the sake of brevity, the term "processor" will henceforth be used to mean "at least one hardware processor," and these terms are used interchangeably.

[0054] Optionally, processor 211 is connected to data cache 210. Optionally, data cache 210 includes multiple data items. Optionally, data cache 210 is stored in a first storage device 201. Optionally, data cache 210 is stored in a second storage device 202. Optionally, data cache 210 is stored in another memory (not shown) connected to processor 211. Optionally, processor 211 may use cache management list 100 to manage data cache 210. Optionally, cache management list 100 includes a tail portion 103 that records multiple recently accessed data items stored in data cache 210. Optionally, tail portion 103 is stored in the first storage device 201. Optionally, cache management list 100 includes a body portion 102 that records multiple less recently accessed data items stored in data cache 210. Optionally, body portion 102 is stored in the second storage device 202. Optionally, the cache management list 100 includes a header portion 101 that records multiple least recently accessed data items stored in the data cache 210. Optionally, the header portion 101 is stored in the first storage device 201.

[0055] Optionally, the processor 211 is connected to one or more digital communication network interfaces 205. For brevity, the term "network interface" is used to refer to "one or more digital communication network interfaces," and these terms are used interchangeably. Optionally, the network interface 205 is connected to a local area network (LAN), such as a wireless LAN or an Ethernet LAN. Optionally, the network interface 205 is connected to a wide area network (WAN), such as a cellular network or the Internet. Optionally, a first storage device 201 is connected to the processor 211 via the network interface 205. Optionally, a second storage device 202 is connected to the processor 211 via the network interface 205.

[0056] Optionally, another processor 220 may be connected to processor 211 via network interface 205. Optionally, the other processor 220 may access data cache 210 by sending one or more access requests to processor 211.

[0057] In order to manage the data cache 210, in some embodiments, the system 200 implements the following optional methods.

[0058] Still referencing Figure 3The diagram illustrates a flowchart schematically representing an optional flow of a method 300 for managing a cache according to some embodiments. In such an embodiment, at 301, processor 211 receives one or more data access requests. Optionally, at least one of the one or more data access requests is received from a software object (e.g., an application) executed by processor 211. Optionally, at least one other data access request is received from another processor 220 via network interface 205.

[0059] In step 302, the processor 211 optionally records one or more data access requests in the tail portion 103. Optionally, recording one or more data access requests in the tail portion 103 includes updating an entry in the tail portion 103, wherein the entry records another access request for a corresponding data item recorded by that entry. Optionally, recording one or more data access requests in the tail portion 103 also includes moving the updated entry to the end of the tail portion 103. Optionally, recording one or more data access requests in the tail portion 103 includes optionally adding a new entry to the tail portion 103 at the end of the tail portion 103.

[0060] Over time, synchronization between the main body portion 102 and the tail portion 103 may be required. In step 311, the processor 211 optionally identifies multiple duplicate entries present in the main body portion 102 and the tail portion 103. In step 312, the processor 211 optionally removes each of the multiple duplicate entries from the main body portion 102 in the second storage device 202 according to the physical organization of the multiple duplicate entries in the second storage device 202.

[0061] Optionally, steps 301, 302, 311, and 312 are executed in each of the multiple management iterations. Optionally, at least some of the multiple management iterations are executed periodically. Optionally, one or more of the multiple management iterations are executed upon identification that the tail portion 103 is full. Optionally, in one or more of the multiple management iterations, in step 313, the processor 211 copies the tail portion 103 to the second storage device 202, for example, when the second storage device 202 is non-volatile, in order to create a persistent copy of the tail portion 103. Optionally, the body portion 102 is stored in the second storage device 202 as multiple files. Optionally, copying the tail portion 103 to the second storage device 202 includes adding the file including the tail portion 103 to the multiple files.

[0062] In at least one of the multiple management iterations, at least one of one or more data access requests can result in a cache miss, wherein at least one data access request is a request to access a data item that is not stored in data cache 210.

[0063] Still referencing Figure 4 The diagram illustrates a flowchart schematically representing an optional flow for a cache miss operation 400 according to some embodiments. In such an embodiment, at 401, processor 211 identifies a data access request in one or more data access requests that requests access to a data item not stored in data cache 210. Optionally, at 402, processor 211 inserts the data item into the data cache. Optionally, to record the data access request, one or more data items need to be evicted from data cache 210. Optionally, at 411, processor 211 identifies one or more entries in header portion 101 that are not in tail 103, wherein one or more entries record access to at least one data item stored in data cache 210. For example, processor 211 may check whether the entry in header portion 101 for recording the first accessed data item is in tail portion 103, and continue in ascending order of access time until one or more entries in header portion 101 that are not in tail portion 103 are identified. Optionally, one or more entries in the header portion 101 are not in the tail portion 103 and not in the body portion 102. In 413, the processor 211 optionally removes one or more entries from the header portion 101.

[0064] Now refer to it again Figure 3 It may be necessary to update the header portion 101 stored in the first storage device 201. In at least some of the multiple management iterations, at 321, the processor 211 optionally identifies a plurality of new least recent entries in the body portion 102, and at 322, the processor 211 optionally copies the plurality of new least recent entries from the body portion 102 in the second storage device 202 to the header portion 101 in the first storage device 201. Optionally, while copying the plurality of new least recent entries from the body portion 102 to the header portion 101, the processor 211 additionally removes the plurality of new least recent entries from the body portion 102, thereby effectively moving the plurality of new least recent entries from the body portion 102 to the header portion 101. Optionally, the processor 211 removes the plurality of new least recent entries from the body portion 102 in another of the multiple management iterations. Optionally, if the processor 211 fails to identify one or more entries in the head portion 101 that are not in the tail portion 103 in 411, at least some of the multiple management iterations may be performed, for example, when the head portion 101 is empty.

[0065] In some embodiments, system 200 manages access to multiple data items, such as multiple data items stored in a cloud-based object storage service. In this example, a bulk of the multiple data items is stored in a storage device remote from processor 211. In such an embodiment, data cache 210 stores multiple copies of each of at least some of the multiple data items. Optionally, the copies stored in data cache 210 include more than 100 terabytes of storage. Optionally, data cache 210 is stored in one or more HDDs, and additionally or alternatively, in one or more SSDs. Optionally, the second storage device 202 of the storage body portion 102 is one or more other HDDs, and additionally or alternatively, is one or more other SSDs. Optionally, the first storage device 201 of the storage head portion 101 and tail portion 103 is a memory component coupled to processor 211.

[0066] In some embodiments, in order to manage access to multiple data items, system 200 implements the following non-mandatory methods.

[0067] Still referencing Figure 5 The diagram illustrates a flowchart schematically representing an optional flow of operation 500 for managing access to multiple data items according to some embodiments. In such an embodiment, at 501, processor 211 optionally manages data cache 210 using method 300. At 510, processor 211 optionally receives one or more data access requests, for example, from another processor 220 or from a software object executed by processor 211. Optionally, the one or more data access requests are for requesting access to at least one of the multiple data items. At 511, processor 211 optionally identifies at least one data item in data cache 210, i.e., at least one data item among at least some copies of the multiple data items stored in data cache 210. At 512, processor 211 optionally accesses at least one data item in data cache 210 in response to one or more data access requests.

[0068] Various embodiments have been described for illustrative purposes, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, their practical application, or improvements to existing technologies in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

[0069] It is anticipated that many related storage technologies will be developed during the lifetime of the patent expiring from this application, and the scope of the term "storage" is intended to a priori include all such new technologies. Furthermore, it is anticipated that many related data items and cache entries will be developed during the lifetime of the patent expiring from this application, and the scope of the terms "data item" and "entry" is intended to a priori include all such new technologies.

[0070] As used in this article, the term “about” means ±10%.

[0071] The terms “comprises,” “comprising,” “includes,” “including,” “having,” and their variations mean “including but not limited to.” This term includes the terms “consisting of” and “consisting essentially of.”

[0072] The phrase “consistently of…” means that a composition or method may include additional ingredients and / or steps, but only if the additional ingredients and / or steps do not substantially alter the essential and novel characteristics of the claimed composition or method.

[0073] As used herein, the singular forms “a,” “an,” and “the” include plural referents unless the context clearly indicates otherwise. For example, the terms “a compound” or “at least one compound” can include multiple compounds, including mixtures thereof.

[0074] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any embodiment described as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments and / or to exclude combinations of features from other embodiments.

[0075] As used herein, the term "optionally" means "provided in some embodiments but not in others." Any particular embodiment may include multiple "optional" features unless these features conflict.

[0076] In this application, various embodiments may be presented in a range format. It should be understood that the range format description is for convenience and brevity only and should not be construed as an inflexible limitation on the range of embodiments. Therefore, the range description should be considered to specifically disclose all possible subranges and individual numerical values ​​within that range. For example, a description of a range such as 1-6 should be considered to specifically disclose subranges such as 1-3, 1-4, 1-5, 2-4, 2-6, 3-6, etc., and individual numerical values ​​within that range such as 1, 2, 3, 4, 5, and 6, regardless of the width of the range.

[0077] Whenever a numerical range is indicated here, it means any referenced number (fraction or integer) included within the indicated range. The phrases “ranging / ranges between the first indicated number and the second indicated number” and “ranging / ranges from the first indicated number to the second indicated number” are used interchangeably herein and are intended to include the first and second indicated numbers as well as all fractions and integers in between.

[0078] It should be understood that certain features of embodiments described in the context of a single embodiment for clarity may also be provided in combination in a single embodiment. Conversely, for brevity, various features of embodiments described in the context of a single embodiment may also be provided individually or in any suitable sub-combination or as suitably as in any other described embodiment. Certain features described in the context of various embodiments are not considered essential features of those embodiments unless the embodiment would be inoperable without those elements.

[0079] Although embodiments have been described in conjunction with specific examples, it will be apparent to those skilled in the art that many alternatives, modifications, and variations will be readily apparent. Therefore, the invention is intended to encompass all such alternatives, modifications, and variations falling within the spirit and broad scope of the appended claims.

[0080] All publications, patents, and patent applications mentioned in this specification are incorporated herein by reference in their entirety, to the extent that each individual publication, patent, or patent application is specifically and individually indicated to be incorporated herein by reference. Furthermore, any reference or designation of any reference in this application should not be construed as an admission that such reference is prior art to the invention. The use of partial headings should not be construed as necessarily limiting. Additionally, any priority documents of this application are incorporated herein by reference in their entirety.

[0081] Embodiments of possible hardware and software environments for the software and / or methods according to the present invention will now be described in detail with reference to the accompanying drawings. Figure 6 This is a functional block diagram showing the various parts of a networked computer system 600, including: a client computer 602; a communication network 604; a storage subsystem 606; a server computer 608; a communication unit 610; a processor set 612; an input / output (I / O) interface set 614; a display device 616; an external device 618; a memory 620; a random access memory (RAM 622); a cache 624; a persistent storage device 626; and a cache manager 628.

[0082] Storage subsystem 606 represents various computer subsystems in this invention in many respects. Therefore, several parts of storage subsystem 606 will now be discussed in the following paragraphs.

[0083] Storage subsystem 606 may be a laptop computer, tablet computer, netbook computer, personal computer (PC), desktop computer, personal digital assistant (PDA), smartphone, or any programmable electronic device capable of communicating with client subsystem via communication network 604. Cache manager 628 is a cluster of machine-readable instructions and / or data for creating, managing, and controlling certain software functions, which will be discussed in detail in the exemplary embodiments subsection of this detailed description section.

[0084] Storage subsystem 606 can communicate with other computer subsystems via communication network 604. Communication network 604 can be, for example, a local area network (LAN), a wide area network (WAN) such as the Internet, or a combination of both, and can include wired, wireless, or fiber optic connections. Typically, communication network 604 can be any combination of connections and protocols supporting communication between server and client subsystems.

[0085] The storage subsystem 606 is shown as a block diagram with numerous double-headed arrows. These double-headed arrows (without separate reference numerals) represent a communication structure that provides communication between the various components of the storage subsystem 606. This communication structure can be implemented using any architecture designed to transfer data and / or control information between processors (such as microprocessors, communication and network processors, etc.), system memory, peripheral devices, and any other hardware components within the system. For example, the communication structure can be implemented at least partially using one or more buses.

[0086] Memory 620 and persistent storage device 626 are computer-readable storage media. Typically, memory 620 may include any suitable volatile or non-volatile computer-readable storage medium. It should also be noted that now and / or in the near future: (i) external device 618 may provide some or all of the memory for storage subsystem 606; and / or (ii) devices external to storage subsystem 606 may provide memory for storage subsystem 606.

[0087] The cache manager 628 is typically stored in persistent storage device 626 via one or more memories of memory 620 for access and / or execution by one or more computer processors in the respective computer processor group 612. Persistent storage device 626: (i) is at least more persistent than a signal in transit; (ii) stores programs (including their soft logic and / or data) on a tangible medium (such as a magnetic or optical domain); and (iii) has significantly less persistence than permanent storage devices. Alternatively, data storage devices may be more persistent and / or permanent than the type of storage provided by persistent storage device 626.

[0088] The cache manager 628 may include machine-readable and executable instructions and / or substantial data (i.e., the type of data stored in the database). In this particular embodiment, persistent storage device 626 includes a magnetic hard disk drive. As some possible variations, persistent storage device 626 may include a solid-state hard disk drive, a semiconductor storage device, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), flash memory, or any other computer-readable storage medium capable of storing program instructions or digital information.

[0089] The media used in persistent storage device 626 can also be removable. For example, a removable hard disk drive can be used in persistent storage device 626. Other examples include optical discs and disks, thumb drives and smart cards, which are inserted into the drive for transfer to another computer-readable storage medium that is also part of persistent storage device 626.

[0090] In these examples, communication unit 610 provides communication with devices external to other data processing systems or storage subsystem 606. In these examples, communication unit 610 includes one or more network interface cards. Communication unit 610 can provide communication by using one or both of physical and wireless communication links. Any software modules discussed herein can be downloaded to persistent storage devices (such as persistent storage device 626) via communication units (such as communication unit 610).

[0091] I / O interface set 614 allows data input and output to other devices, which can be locally connected to the server computer 608 for data communication. For example, I / O interface set 614 provides connectivity to external device 618. External device 618 may include devices such as a keyboard, keypad, touchscreen, and / or other suitable input devices. External device 618 may also include portable computer-readable storage media such as thumb drives, portable optical discs or disks, and memory cards. Software and data used to implement embodiments of the invention, such as cache manager 628, may be stored on such portable computer-readable storage media. In these embodiments, the associated software may (or may not) be loaded, in whole or in part, onto persistent storage device 626 via I / O interface set 614. I / O interface set 614 also connects to display device 616 for data communication.

[0092] Display device 616 provides a mechanism for displaying data to a user and can be, for example, a computer monitor or a smartphone display screen.

[0093] The programs described herein are identified based on applications that implement them in specific embodiments of the invention. However, it should be understood that any particular program terminology used herein is for convenience only, and therefore the invention should not be limited to use only in any particular application identified and / or implied by such terminology.

[0094] Various embodiments of the invention have been described for illustrative purposes, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, their practical application, or improvements to existing technologies in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

[0095] All the means or steps plus functional elements in the following claims are intended to include any structure, material, action, and equivalent for performing a function in combination with other claimed elements as specifically claimed. The description of this disclosure has been presented for purposes of illustration and description, but it is not intended to be exhaustive or limited to the form of disclosure disclosed. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of this disclosure. The embodiments were chosen and described in order to best explain the principles and practical application of this disclosure and to enable others skilled in the art to understand the disclosure of various embodiments with various modifications suitable for the particular intended use.

Claims

1. A computer-implemented method for managing a data cache, comprising: The cache management list stores multiple entries, including: The tail portion, stored in the first storage device, records multiple recently accessed data items stored in the data cache. The main body, stored in the second storage device, records multiple less recently accessed data items stored in the data cache, and The header portion, stored in the first storage device, records a first plurality of least recently accessed data items stored in the data cache; In at least one cache management iteration out of multiple cache management iterations: Identify the second or more least recent entries in the main body. Record the second or more least recently accessed data items stored in the data cache, and Copy the second set of least recent entries from the body portion to the header portion; In each iteration of cache management across multiple cache management iterations: Receive the first data access request; The first data access request is recorded in the tail portion; Identify multiple duplicate entries associated with the first data access request existing in the body portion and the tail portion; and Based on the physical organization of the plurality of duplicate entries in the second storage device, each of the plurality of duplicate entries is removed from the main portion.

2. The method of claim 1, further comprising copying the tail portion to the second storage device.

3. The method according to claim 1, wherein, In at least one of the multiple cache management iterations, the method further includes: Based on the first data access request, identify data items that are not stored in the data cache; and The data item is inserted into the data cache.

4. The method according to claim 3, wherein, In at least one of the multiple cache management iterations, the method further includes: Identify at least one entry present in the head portion but not in the tail portion, the entry recording access to at least one data item stored in the data cache; and Remove the entry from the header section.

5. The method according to claim 1, wherein, Recording the first data access request in the tail portion includes an operation selected from a group comprising: Update the first entry in the tail portion, the first entry recording another access request for a corresponding data item recorded therethrough, and move the updated first entry to the end of the tail portion; and Add a second entry to the tail section.

6. The method according to claim 1, wherein, At least one cache management iteration in the multiple cache management iterations is executed periodically.

7. The method according to claim 1, further comprising: It has been determined that the tail portion is full; as well as In response to determining that the tail portion is full, at least one cache management iteration of the multiple cache management iterations is executed.

8. The method according to claim 4, further comprising: Determine that at least one entry present in the head portion is not included in the tail portion; In response to determining that at least one entry present in the header portion is not included in the tail portion, at least one cache management iteration of the plurality of cache management iterations is performed.

9. The method according to claim 1, wherein, The ratio between the first access time of the first storage device and the second access time of the second storage device is less than the access time ratio threshold.

10. The method according to claim 1, wherein: The first storage device is selected from a first group of digital storage devices, the first group of digital storage devices including: Random access memory (RAM), static RAM (SRAM), first dynamic RAM (DRAM), first hard disk drive, and first solid-state drive; and The second storage device is selected from a second group of digital storage devices, which includes: First DRAM, second DRAM, first hard disk drive, second hard disk drive, first solid-state drive, second solid-state drive, electrically erasable programmable read-only memory (EEPROM), NAND flash memory, network-connected storage device, and network memory.

11. The method according to claim 2, wherein: The main body is stored as multiple files in the second storage device; and Copying the tail portion to the second storage device includes adding a file including the tail portion to the plurality of files.

12. A computer system for managing a data cache, comprising: The processor set includes a first processor and a second processor; as well as One or more computer-readable storage media; in: The processor set is constructed, positioned, linked, and / or programmed to execute program instructions commonly stored on the one or more computer-readable storage media; and When the program instructions are executed by the processor set, the processor set manages the data cache through the following steps: The cache management list stores multiple entries, including: The tail portion is stored in a first storage device connected to the first processor, recording multiple recently accessed data items stored in the data cache. The main body is stored in a second storage device connected to the first processor, recording multiple less recently accessed data items stored in the data cache, and The header portion, stored in the first storage device, records a first plurality of least recently accessed data items stored in the data cache. In at least one cache management iteration out of multiple cache management iterations: Identify the second or more least recent entries in the main body; Record the second or more least recently accessed data items stored in the data cache; and Copy the second set of least recent entries from the body portion to the header portion. In each iteration of cache management across multiple cache management iterations: Receive the first data access request; The first data access request is recorded in the tail portion; Identify multiple duplicate entries associated with the first data access request existing in the body portion and the tail portion; and Based on the physical organization of the plurality of duplicate entries in the second storage device, each of the plurality of duplicate entries is removed from the main portion.

13. The computer system of claim 12, further comprising instructions programmed to perform the following operations: The data access request is received from the at least one data access request from the software object executed by the first processor.

14. The computer system of claim 12, further comprising instructions programmed to perform the following operations: The data access request is received from the at least one data access request from the second processor.

15. The computer system according to claim 14, wherein, The second processor is connected to the first processor via at least one digital communication network interface connected to the first processor.

16. The computer system according to claim 12, wherein, The computer system includes computer program products.

17. A computer system for managing access to multiple data items, comprising: Processor set; as well as One or more computer-readable storage media; in: The processor set is constructed, positioned, linked, and / or programmed to execute program instructions commonly stored on the one or more computer-readable storage media; and When executed by the processor set, the program instructions cause the processor set to manage access to multiple data items through the following steps: The first data item is stored in the data cache; The cache management list stores multiple entries, including: The tail portion is stored in the first storage device, recording multiple recently accessed data items stored in the data cache. The main body, stored in the second storage device, records multiple less recently accessed data items stored in the data cache, and The header portion, stored in the first storage device, records multiple least recently accessed data items stored in the data cache. Perform data access management operations, including: Receive the first data access request; The first data access request is recorded in the tail portion; Identify multiple duplicate entries existing in the body portion and the tail portion in response to the first data access request; and Based on the physical organization of the plurality of duplicate entries in the second storage device, each of the plurality of duplicate entries is removed from the main portion.

18. The computer system of claim 17, further comprising instructions programmed to perform the following operations: Receive a second data access request, requesting access to the first data item; In response to receiving the second data access request: Identify the first data item stored in the data cache; and Access the first data item.

19. The computer system according to claim 17, wherein, The computer system includes computer program products.

Images