System and method for cache allocation

Abstract

A method for data storage includes receiving, by a storage device, a read request for data, the read request associated with a request size, determining that a first cache region associated with a first portion of the data is in a first portion of a cache, the first cache region having a first size less than the request size, a second cache region associated with a second portion of the data is determined to be in the second portion of the cache, the second cache region having a second size that is less than the request size and different in size than the first size, and based on the read request, the second cache region is associated with the second portion of the data. A first portion of the data is read from the first cache region and a second portion of the data is read from the second cache region.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of U.S. Provisional Application Serial No. 63 / 527,908, filed on July 20, 2023, entitled “MEMORY EFFICIENT CACHE MANAGEMENT WITH ADAPTIVE CACHE LINE SIZE,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] Aspects of some embodiments of the present disclosure relate to systems and methods for cache region allocation. Background Art

[0004] In the field of computer storage, a system may include a host computer and one or more storage devices connected to (e.g., communicatively coupled to) the host computer. Such computer storage systems have become increasingly popular, in part, to allow many different users to share the computing resources of the system. As the number of users of such systems and the number and complexity of applications running on such systems have increased, storage requirements have also increased over time.

[0005] Therefore, there may be a need for methods, systems, and devices suitable for improving the use of storage devices in storage systems.

[0006] This Background section is intended only to provide context, and the disclosure of any embodiment or concept in this section does not constitute an admission that the embodiment or concept is prior art. Summary of the invention

[0007] Aspects of some embodiments of the present disclosure relate to computer storage systems and provide improvements to cache area allocation (e.g., cache block allocation) by reducing cache pollution and bandwidth pollution. As used herein, "pollution" refers, at least in part, to wasted resources of a storage system.

[0008] According to some embodiments of the present disclosure, a method for data storage is provided, the method comprising receiving, by a storage device, a read request for data, the read request being associated with a request size, determining that a first cache area associated with a first portion of the data is in a first portion of the cache, the first cache area having a first size that is smaller than the request size, determining that a second cache area associated with a second portion of the data is in a second portion of the cache, the second cache area having a second size that is smaller than the request size and different from the first size, and based on the read request, reading the first portion of the data from the first cache area and reading the second portion of the data from the second cache area.

[0009] The first portion of the cache may include a first cache group divided into cache areas having a first size, and the second portion of the cache may include a second cache group divided into cache areas having a second size.

[0010] Determining that the first cache region is in the first portion of the cache may include checking a first lookup data structure associated with the first size, and determining that the second cache region is in the second portion of the cache may include checking a second lookup data structure associated with the second size.

[0011] The first lookup data structure may include a first key-value store, and the second lookup data structure may include a second key-value store.

[0012] The first portion of the cache may include a first cache line and the second portion of the cache may include a second cache line, the first cache line being separate from the second cache line.

[0013] The first cache area and the second cache area may be linked by a first least recently used (LRU) list.

[0014] The first LRU list may include a group LRU list, and the first cache area and the second cache area may be linked by a second LRU list different from the first LRU list.

[0015] According to some other embodiments of the present disclosure, a storage device including a cache and a processing circuit is provided, wherein the processing circuit is configured to receive a read request for data, the read request being associated with a request size, determine a first cache area associated with a first portion of the data in a first portion of the cache, the first cache area having a first size smaller than the request size, determine a second cache area associated with a second portion of the data in a second portion of the cache, the second cache area having a second size smaller than the request size and different from the first size, and based on the read request, read the first portion of the data from the first cache area and read the second portion of the data from the second cache area.

[0016] The first portion of the cache may include a first cache group divided into cache areas having a first size, and the second portion of the cache may include a second cache group divided into cache areas having a second size.

[0017] The processing circuitry may be configured to determine that the first cache region is in a first portion of the cache by checking a first lookup data structure associated with the first size, and to determine that the second cache region is in a second portion of the cache by checking a second lookup data structure associated with the second size.

[0018] The first lookup data structure may include a first key-value store, and the second lookup data structure may include a second key-value store.

[0019] The first portion of the cache may include a first cache line and the second portion of the cache may include a second cache line, the first cache line being separate from the second cache line.

[0020] The first cache area and the second cache area may be linked by a first least recently used (LRU) list.

[0021] The first LRU list may include a group LRU list, and the first cache area and the second cache area may be linked by a second LRU list different from the first LRU list.

[0022] According to some other embodiments of the present disclosure, a system for data storage is provided, including a processor and a memory storing instructions, which instructions, based on being executed by the processor, cause the processor to execute: receiving a read request for data, the read request being associated with a request size, determining that a first cache area associated with a first portion of the data is in a first portion of the cache, the first cache area having a first size that is smaller than the request size, determining that a second cache area associated with a second portion of the data is in a second portion of the cache, the second cache area having a second size that is smaller than the request size and different from the first size, and based on the read request, reading the first portion of the data from the first cache area and reading the second portion of the data from the second cache area.

[0023] The first portion of the cache may include a first cache group divided into cache areas having a first size, and the second portion of the cache may include a second cache group divided into cache areas having a second size.

[0024] Determining that the first cache region is in the first portion of the cache may include checking a first lookup data structure associated with the first size, and determining that the second cache region is in the second portion of the cache may include checking a second lookup data structure associated with the second size.

[0025] The first lookup data structure may include a first key-value store, and the second lookup data structure may include a second key-value store.

[0026] The first portion of the cache may include a first cache line and the second portion of the cache may include a second cache line, the first cache line being separate from the second cache line.

[0027] The first cache area and the second cache area may be linked by a first least recently used (LRU) list and by a second LRU list different from the first LRU list. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Non-limiting and non-exhaustive embodiments of the present disclosure are described with reference to the following figures, wherein like reference numerals refer to like parts throughout the various views unless otherwise specified.

[0029] Figure 1 is a system diagram depicting a system for adaptive cache block allocation according to some embodiments of the present disclosure.

[0030] Figure 2 is a diagram depicting an example of adaptive cache block allocation according to some embodiments of the present disclosure.

[0031] Figure 3 is a system diagram depicting an example of set-based cache organization and two-level cache replacement in a system for adaptive cache block allocation according to some embodiments of the present disclosure.

[0032] Figure 4 is a flow chart depicting example operations of a method for adaptive cache block allocation according to some embodiments of the present disclosure.

[0033] In the several views of the accompanying drawings, corresponding reference numerals indicate corresponding components. It will be appreciated by those skilled in the art that the elements in the accompanying drawings are shown for simplicity and clarity and are not necessarily drawn to scale. For example, the sizes of some elements, layers, and regions in the drawings may be exaggerated relative to other elements, layers, and regions to help improve the understanding and clarity of the various embodiments. In addition, common but well-known elements and parts that are not related to the description of the embodiments may not be shown to facilitate less obstructed views of these various embodiments and to make the description clear. DETAILED DESCRIPTION

[0034] By referring to the detailed description and the accompanying drawings of one or more embodiments, it is easier to understand the aspects of the present disclosure and the methods for implementing the present disclosure. Hereinafter, the embodiments will be described in more detail with reference to the accompanying drawings. However, the described embodiments may be embodied in various different forms and should not be construed as being limited to the embodiments shown herein. On the contrary, these embodiments are provided as examples so that the present disclosure will be thorough and complete, and will fully convey the various aspects of the present disclosure to those skilled in the art. Therefore, the description of processes, elements and techniques that are unnecessary for a complete understanding of the various aspects and features of the present disclosure for those of ordinary skill in the art can be omitted.

[0035] Unless otherwise specified, the same reference numerals, characters, or combinations thereof represent the same elements throughout the drawings and written descriptions, and therefore their descriptions will not be repeated. In addition, for clarity of description, parts not related to the description of the embodiments may not be shown. In the drawings, the relative sizes of elements, layers, and regions may be exaggerated for clarity.

[0036] In the detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the various embodiments. However, it is apparent that the various embodiments can be practiced without these specific details or with one or more equivalent arrangements.

[0037] It should be understood that although the terms "zeroth", "first", "second", "third", etc., may be used herein to describe various elements, components, regions, layers and / or parts, these elements, components, regions, layers and / or parts should not be limited by these terms. These terms are used to distinguish one element, component, region, layer or part from another element, component, region, layer or part. Therefore, without departing from the spirit and scope of the present disclosure, the first element, component, region, layer or part described below may be referred to as a second element, component, region, layer or part.

[0038] It will be understood that when an element or component is referred to as being "on another element or component," "connected to another element or component," or "coupled to another element or component," it may be directly on, connected to, or coupled to another element or component, or there may be one or more intermediate elements or components. However, "direct connection / direct coupling" refers to a component being directly connected or coupled to another component without the need for intermediate components. At the same time, other expressions describing the relationship between components, such as "between," "immediately between," or "adjacent to," and "directly adjacent to" may be similarly interpreted. In addition, it will be understood that when an element or component is referred to as being "between" two elements or components, it may be the only element or component between the two elements or components, or there may also be one or more intermediate elements or components.

[0039] The terms used herein are only used for the purpose of describing specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a" and "an" are also intended to include plural forms unless the context clearly indicates otherwise. It will also be understood that the terms "comprises", "comprising", "have", "having", "includes" and "including" when used in this specification specify the presence of the features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. As used herein, each term "or" and "and / or" includes any and all combinations of one or more associated listed items.

[0040] For the purposes of this disclosure, expressions such as "at least one of" when following a list of elements modify the entire list of elements, rather than the individual elements in the list. For example, "at least one of X, Y, or Z," "at least one of X, Y, and Z," and "at least one selected from the group consisting of X, Y, and Z" may be interpreted as any combination of only X, only Y, only Z, or two or more of X, Y, and Z, such as, for example, XYZ, XYY, YZ, and ZZ.

[0041] As used herein, the terms "substantially," "approximately," "approximately," and similar terms are used as terms of approximation rather than terms of degree, and are intended to account for the inherent deviations in measured or calculated values ​​that one of ordinary skill in the art would recognize. As used herein, "approximately" or "approximately" includes the value being described and means, taking into account the measurement in question and the errors associated with the measurement of the particular quantity (i.e., the limitations of the measurement system), within an acceptable range of deviations for the particular value determined by one of ordinary skill in the art. For example, "approximately" can mean within one or more standard deviations, or within ±30%, 20%, 10%, 5% of the value being described. In addition, the use of "may" when describing embodiments of the present disclosure refers to "one or more embodiments of the present disclosure."

[0042] When one or more embodiments can be implemented differently, the specific process order can be performed in a different order than the described order. For example, two processes described in succession can be performed substantially simultaneously or in an order opposite to the described order.

[0043] Any component or any combination of components described (e.g., in any system diagram included herein) may be used to perform one or more operations of any flow diagram included herein. In addition, (i) the operations are merely examples and may involve various additional operations not explicitly included, and (ii) the temporal order of the operations may vary.

[0044] Any suitable hardware, firmware (e.g., an application specific integrated circuit), software, or a combination of software, firmware, and hardware may be used to implement the electronic or electrical devices and / or any other related devices or components according to the embodiments of the present disclosure described herein. For example, the various components of these devices may be formed on an integrated circuit (IC) chip or on separate IC chips. In addition, the various components of these devices may be implemented on a flexible printed circuit film, a tape carrier package (TCP), a printed circuit board (PCB), or formed on a substrate.

[0045] In addition, the various components of these devices can be processes or threads running on one or more processors in one or more computing devices, executing computer program instructions and interacting with other system components to perform the various functions described herein. The computer program instructions are stored in a memory that can be implemented in a computing device using a standard storage device, such as, for example, a random access memory (RAM). The computer program instructions can also be stored in other non-transitory computer-readable media, such as, for example, a CD-ROM, a flash drive, etc. In addition, those skilled in the art should recognize that the functions of various computing devices can be combined or integrated into a single computing device, or the functions of a particular computing device can be distributed on one or more other computing devices without departing from the spirit and scope of the embodiments of the present disclosure.

[0046] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the inventive concept belongs. It will also be understood that terms (such as those generally defined in dictionaries) should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and / or this specification, and should not be interpreted in an idealized or overly formal sense, unless explicitly defined as such in this document.

[0047] As described above, in the field of computer storage, a system may include a host and one or more storage devices (eg, solid state drives (SSDs)) communicatively coupled to the host. The storage devices may store data associated with applications running on the host.

[0048] Storage devices may include caches and storage. Caches may include cache lines. Caches may be used as part of a memory (e.g., dynamic random access memory (DRAM)) to access and process data faster than with storage. For example, storage may include non-volatile memory (NVM) that stores data in a manner that takes longer to access and process than data stored in the memory. Cache lines may include copies of data that has recently been accessed from storage (e.g., data that has been accessed more recently than some data in the storage).

[0049] Conventionally, cache lines of a storage device may be arranged based on a fixed cache block size (e.g., 16 kilobytes (KiB)). For example, in response to an input / output (I / O) request, data copied from a storage device may be placed in a cache line according to an offset and request size (e.g., the requested size of data associated with the I / O request) and according to the fixed cache block size of the cache line. Such fixed size cache block systems may experience problems with cache pollution and bandwidth pollution.

[0050] Aspects of one or more embodiments of the present disclosure provide improvements to cache management by providing adaptive cache region allocation (eg, cache block allocation), which may reduce cache pollution and bandwidth pollution.

[0051] Different applications running on a host may have various workloads with different I / O patterns. For example, in some systems, approximately 60% of requests from a first application may have a request size of 4KiB or less, while approximately 50% of requests from a second application may have a request size of approximately 32KiB or greater. Applications that process workloads with I / O requests associated with small request sizes, such as transactional databases, may perform better using small cache block sizes. For example, using a small cache block size for small request sizes may result in less cache pollution and less bandwidth pollution. On the other hand, applications that process workloads with large request sizes, such as multimedia systems, may perform better using large cache block sizes. For example, using a large cache block size for large I / O requests may result in lower I / O counts (e.g., the number of I / Os used to process a given request).

[0052] Each cache block may require approximately 40 bytes of metadata. For example, the metadata may include identification information and location information for finding data associated with a given request in a storage device. For example, the identification information and location information may include information stored in a lookup data structure, such as a key and value of a key-value (KV) store. Using a large number of small cache block sizes may result in larger memory usage for each block and their corresponding metadata. For example, for a 384 terabyte (TiB) cache, a commonly used 16KiB cache block size may result in 960 gigabytes (GiB) of metadata in memory. On the other hand, using a small number of large cache block sizes may consume less memory than using a large number of small cache block sizes, but may result in more cache pollution and more bandwidth pollution.

[0053] Aspects of one or more embodiments of the present disclosure provide improvements in cache management by storing data in a cache of a storage device having cache blocks of different sizes based on a given request size.

[0054] In some embodiments, as discussed in further detail below, the storage device can generate cache miss intervals by: maintaining (e.g., maintaining) a key-value store for each cache block size (e.g., 32KiB, 64KiB, 128KiB, and 256KiB); aligning the I / O offset of the I / O request with each cache block size; searching from the smallest cache block size to avoid inconsistencies; increasing the start offset by the cache block size; and terminating when the start offset meets the end offset.

[0055] In some embodiments, the storage device may perform "greedy" cache block allocation by: starting with a maximum cache block size to ensure that a minimum number of cache blocks are used for cache block allocation; checking alignment of each I / O offset; and terminating when the starting I / O offset meets the ending I / O offset.

[0056] In some embodiments, the storage device may divide the cache space into discrete pieces for grouping.

[0057] In some embodiments, a storage device may perform a group-based cache organization to reduce problematic cache fragmentation by placing cache blocks of the same size into groups (e.g., groups of the same size); storing cache blocks in the same group physically adjacently; and replacing an entire group when the cache is full.

[0058] In some embodiments, the storage device can perform a two-level cache replacement scheme to improve the cache hit rate by: maintaining two least recently used (LRU) lists (a global LRU list that links all cache blocks in the cache and a group LRU list that links all cache groups in the cache); and if the newly allocated cache block size is the same as the cache block size of the tail of the global LRU list (e.g., the oldest cache block), then replacing the tail, or replacing the tail of the group LRU list, to free up space in the cache.

[0059] Thus, the present disclosure provides a storage system with improved cache performance by: adjusting cache line size (e.g., cache block size) based on workload characteristics; reducing memory footprint of large-scale cache systems; providing flexibility to work with various cache scenarios; and running fast enough to accommodate online applications.

[0060] Figure 1 is a system diagram depicting a system for adaptive cache block allocation according to some embodiments of the present disclosure.

[0061] refer to Figure 1, the system 1 may include a host 100 and a storage device 200 communicatively coupled to the host 100. The host 100 may include one or more applications 111 running on computing resources of the host 100. For example, the host may include a first application 111a processing a first workload WLa and a second application 111b processing a second workload WLb. In order to process their respective workloads, the applications 111 may send requests to access data stored at the storage device 200. For example, the first application 111a and the second application 111b may send respective I / O requests (e.g., read requests) 10 to the storage device 200.

[0062] The storage device 200 may include a cache 240 and a storage device 250. The cache 240 may include one or more cache lines CL for storing a cache area (e.g., a cache block CB including a zeroth cache block CB0 to a fourth cache block CB4). In some embodiments, the cache lines CL may be logically separate but physically together (e.g., continuous). In some embodiments, the cache lines CL may be divided into discontinuous fragments (e.g., discontinuous portions). Some cache blocks CB may be different in size from some other cache blocks CB. For example, some cache blocks CB may have a first size S1; some cache blocks CB may have a second size S2; some cache blocks CB may have a third size S3; and some cache blocks CB may have a fourth size S4. For example, the first size S1 may be 32KiB; the second size S2 may be 64KiB; the third size S3 may be 128KiB; and the fourth size may be 256KiB. The cache blocks CB may be associated with data in corresponding data blocks DB in the data blocks DB stored in the storage device 250. For example, the zeroth cache block CB0 may store a copy of the data stored in the zeroth data block DB0.

[0063] The cache 240 may be associated with one or more lookup data structures (e.g., a first lookup data structure 211 to a fourth lookup data structure 214). The lookup data structures may keep track of (e.g., maintain) the location of data stored in the storage device 200. For example, the first lookup data structure 211 may maintain the location of data associated with a cache block CB having a first size S1; the second lookup data structure 212 may maintain the location of data associated with a cache block CB having a second size S2; the third lookup data structure 213 may maintain the location of data associated with a cache block CB having a third size S3; and the fourth lookup data structure 214 may maintain the location of data associated with a cache block CB having a fourth size S4. In some embodiments, the lookup data structure may be a key-value store that stores keys (e.g., first key K1 to nth key Kn, where n is an integer greater than zero) and values ​​(e.g., first value V1 to nth value Vn, where n is an integer greater than zero) for locating requested data.

[0064] In some embodiments, the storage device 200 may receive a read request 10 from the host 100. The read request 10 may be associated with a request size 12 (see Figure 2 ). The request size 12 may correspond to an aligned request range. For example, the read request 10 may include an original request offset and an end address (e.g., the original request offset and information associated with the original request length 11). The storage device 200 may determine the request size 12 by aligning the original request offset and the end address with the minimum block size of the cache 240. In some embodiments, the storage device 200 may check a lookup data structure (e.g., 211-214) for data associated with the read request 10. The storage device 200 may generate a list of missing intervals for all portions of the read request 10 that are missing from the cache 240. The storage device 200 may traverse the read request 10 to find all missing intervals. For example, because the cache 240 uses a variable (e.g., more than one) cache block size, the storage device may check a lookup data structure (e.g., a key-value store in a memory) for each cache block size (e.g., S1-S4) to determine whether any portion of the data associated with the read request 10 is cached (e.g., stored) at each corresponding cache block size.

[0065] Figure 2 is a diagram depicting an example of adaptive cache block allocation according to some embodiments of the present disclosure.

[0066] refer to Figure 2 , a read request 10 may be associated with, for example, an original request offset of 48KiB and an original request length 11 of 184KiB. Cache 240 (see Figure 1 and 3 ) can adopt cache block sizes of, for example, 32KiB, 64KiB, 128KiB, and 256KiB. The latter portion of the data associated with the read request 10 (e.g., from 128KiB to 232KiB) can be cached at a 128KiB block size (e.g., as a third cache block CB3). The request range for request alignment can be from 32KiB to 256KiB, thereby having a request size 12 of 224KiB. Because the original request length 11 associated with the read request 10 in this example is 184KiB, the storage device can determine that the request size 12 based on the offset of 48KiB and the original request length 11 of 184KiB is equal to 224KiB based on the seven smallest cache block sizes (e.g., 32KiB) used to cover (e.g., store) the data associated with the read request 10.

[0067] Within the requested range (e.g., within the corresponding range of the request size 12 of 224KiB and the cache block size), the storage device 200 can search from the minimum cache block size (e.g., 32KiB) and determine whether the current address associated with the read request 10 is cached at any cache block size. The storage device 200 can align the current address to different cache block sizes (e.g., using an equation that relates the aligned offset to the request offset and the cache block size). For example, the storage device 200 can determine aligned offsets of 32KiB, 0, 0, and 0 for cache block sizes of 32KiB, 64KiB, 128KiB, and 256KiB, respectively. The storage device 200 can use the aligned offsets to search a lookup data structure (e.g., a key-value store in a memory) for each corresponding cache block size. If the result of the search of the lookup data structure is a total miss (e.g., a cache miss), the storage device 200 may determine that the current address with the minimum cache block size (e.g., an interval between 32KiB and 64KiB) is not cached. The storage device 200 may add the interval to the list of missing intervals. In some embodiments, in order to allocate the largest possible cache block for the missing interval, the storage device 200 may merge consecutive missing intervals. The storage device 200 may proceed to the next address associated with the read request 10 (e.g., 64KiB), and the process may be repeated. After checking the lookup data structure, the storage device 200 may obtain a complete list of missing intervals. For example, an interval from 32KiB to 128KiB may be missing in the cache 240. For each missing interval in the missing interval list, the storage device 200 may use the largest possible cache block size (e.g., the largest cache block size that fits the missing interval) to allocate a cache block CB. Using the largest possible cache block size to fit the missing interval may be referred to as "greedy allocation." Greedy allocation ensures that the storage device 200 reduces the number of allocated cache blocks CB and I / O counts. To determine whether the cache block size is appropriate for the miss interval, the storage device 200 can determine whether the cache block is within the range of the miss interval, because addresses outside these intervals may have been cached. For example, the storage device 200 can determine how to allocate (e.g., adaptively allocate) cache block sizes for the miss interval from 32KiB to 128KiB by determining that the largest possible cache block size for the miss interval is 32KiB, because larger cache block sizes (64KiB, 128KiB, and 256KiB) will start outside the miss interval. For example, if a 64KiB cache block is used at the beginning (from 32KiB to 96KiB), and then another 64KiB cache block is used in the interval from 96KiB to 128KiB, overlap will occur.For the remaining missing intervals from 64KiB to 128KiB, for this example, the largest possible cache block size is 64KiB, because the interval from 64KiB to 128KiB is within the range of the missing interval (64KiB to 128KiB). At the end of the greedy allocation process, the storage device 200 can cache two cache blocks CB, including a first cache block CB1 having a first size from 32KiB to 64KiB (e.g., 32KiB) and a second cache block CB2 having a second size S2 from 64KiB to 128KiB (e.g., 64KiB).

[0068] Assuming N is the original request length 11, M is the number of different cache block sizes, and K is the total number of cache blocks CB in cache 240, the corresponding greedy cache block allocation algorithm will have a time complexity of O(K*N) and O(K*N*M) for fixed-size and adaptive cache block allocation, respectively. (The O in the equation corresponds to "big O notation"). In some embodiments, M can be set to a constant value, such as Figure 2 As shown in the example of FIG. 4, the time complexity can be approximately O(K*N), which is equivalent to a fixed-size cache block allocation. The space complexity of the algorithm can be the same as a fixed-size cache block allocation (which is O(K)).

[0069] By using both small cache blocks and large cache blocks, adaptive cache block allocation can provide an improvement over fixed-size cache block allocation schemes for handling dynamic workloads (e.g., dynamic cloud workloads). However, adaptive cache block allocation can result in problematic fragmentation. For example, Figure 1 As can be seen in FIG, using small cache blocks CB and large cache blocks CB in the same cache line CL may result in many scattered holes, such as the zeroth cache block CB0, which may be difficult to use for other requests. For example, most request sizes may be larger than the 32KiB cache block size of the zeroth cache block CB0. When the cache 240 becomes full and adaptive cache blocks CB are allocated, the cache space may be divided into discontinuous blocks of variable size. To address problematic fragmentation, some embodiments of the present disclosure may incorporate a group-based cache organization, as described below with reference to FIG. Figure 3 discussed.

[0070] Figure 3 is a system diagram depicting an example of set-based cache organization and two-level cache replacement in a system for adaptive cache block allocation according to some embodiments of the present disclosure.

[0071] refer to Figure 3In some embodiments, the storage device 200 may group cache blocks CB of the same cache block size together in a group (e.g., a group of the same size). For example, cache blocks (e.g., the zeroth cache block CB0 and the first cache block CB1) belonging to the same group (e.g., the first group G1) may be physically stored adjacent to each other in the cache 240. Therefore, when the cache 240 is full, the entire group (e.g., the first group G1) may be replaced to create a continuous segment of cache space for cache block allocation.

[0072] In some embodiments, the storage device 200 can select the largest cache block size (e.g., the fourth size S4) as the group size for each group (e.g., the first group G1 to the fourth group G4). Therefore, replacing the entire group can free up enough cache space for the largest cache block CB allocation. In the case of a small block allocation, replacing the entire group can create an open group that can be used to allocate many cache blocks of that block size.

[0073] As an example of a cache organization based on a group, the cache block size may include a first size S1 of 32KiB, a second size S2 of 64KiB, a third size S3 of 128KiB, and a fourth size S4 of 256KiB. The group sizes may all be 256KiB. The first group G1, the second group G2, and the third group G3 may be open (e.g., may have cache blocks available for allocation). The fourth group G4 may be a full group based on storing a fourth cache block CB4 having a size of 256KiB. In some embodiments, the group sizes may all be equal.

[0074] In order to allocate a cache block CB, the storage device 200 can determine whether the cache 240 is full. If the cache 240 is not full, the storage device 200 can determine whether there is an open group with the same block size as the cache block CB being allocated. For example, a slab allocator (e.g., a software and / or hardware component of a storage system that manages space in groups) can determine whether there is an open group with a suitable cache block size. If there is an open group, a cache block CB with a size suitable for the cache block can be allocated from the open group. On the other hand, if there is no such open group, the storage device can create a new group and cache blocks CB can be allocated from the new group. If the cache 240 is full, the storage device 200 can replace the entire group. For example, the storage device 200 can replace an entire group (e.g., the fourth group G4) to free up enough cache space for the maximum block size. In some embodiments, and as discussed in further detail below, the storage device 200 can replace the entire group based on one or more LRU lists to replace the oldest (e.g., least recently used) group, such as Figure 3 As shown, this group is also the fourth group G4.

[0075] For example, in some embodiments, the storage device 200 may use (e.g., may follow or provide) a group-based LRU replacement policy. In such an embodiment, the first LRU list may include (e.g., may be) a group LRU list 270. In such an embodiment, when a cache block CB is accessed, the group containing the cache block CB may be promoted to the head of the group LRU list 270 (e.g., may be designated as the most recently used cache block CB). When the cache 240 is full, the storage device 200 may replace the group at the tail of the group LRU list 270 (e.g., may replace and free up storage space associated with the least recently used group). Figure 3 , the first cache block CB1 is located at the head of the group LRU list 270, and the fourth cache block CB4 is located at the tail of the group LRU list 270. Although a cache miss may trigger a write-back I / O for the entire group to be evicted, the total I / O volume associated with the write-back I / O may be less than the total I / O volume associated with using a large fixed-size cache block CB. One benefit of evicting an entire group in an adaptive cache block allocation system is that all space in the cache 240 that was previously associated with the group can be freed (e.g., immediately freed) and used to store multiple small cache blocks CB from other (e.g., future) data access requests.

[0076] In some embodiments, the storage device 200 may use (e.g., may follow or provide) a global cache block LRU replacement policy. In such an embodiment, the second LRU list may include (e.g., may be) a global LRU list 280. In some embodiments, the storage device 200 may also include a global LRU list 280 in addition to the group LRU list 270. The global LRU list may allow the storage device 200 to remove infrequently accessed (or "cold") cache blocks CB in the same group as frequently accessed (or "hot") cache blocks CB. Such cold cache blocks CB may cause cache pollution based on running out of space in the cache 240 that may not be used to accommodate more frequently accessed cache blocks CB. As Figure 3 As depicted, the first cache block CB1 is located at the head of the global LRU list 280 , and the fourth cache block CB4 is located at the tail of the global LRU list 280 .

[0077] In some embodiments, all cache blocks CB in cache 240 may be linked (e.g., logically linked) using global LRU list 280. If cache 240 is full, storage device 200 may attempt to allocate a new cache block CB by determining whether the cache block CB at the tail of global LRU list 280 is the same size as the new cache block CB. If the cache block CB at the tail of global LRU list 280 has the same size as the new cache block CB, storage device 200 may replace the cache block CB at the tail of global LRU list 280. Storage device 200 may also promote both the cache block CB at the tail of global LRU list 280 and its group to the head of group LRU list 270 and global LRU list 280. If the cache block CB at the tail of global LRU list 280 is different in size from the new cache block CB, storage device 200 may replace the entire group using a group-based LRU replacement strategy. For example, storage device 200 may replace the group at the tail of group LRU list 270.

[0078] In some embodiments, storage device 200 may use a lock-free design to avoid high lock contention overhead when accessing cache 240 in parallel.

[0079] Figure 4 is a flow chart depicting example operations of a method for adaptive cache block allocation according to some embodiments of the present disclosure.

[0080] refer to Figure 4 , the method 4000 may include the following example operations. The storage device 200 may receive a read request 10 for data (see Figure 1 and 3 )(operation 4001). The read request 10 may be associated with a request size of 12 (operation 4001). The storage device 200 may determine that a first cache block CB1 associated with a first portion of the data is in a first portion of the cache 240 (operation 4002). The first cache block CB1 may have a first size that is smaller than the request size 12 (operation 4002). The storage device 200 may determine that a second cache block CB2 associated with a second portion of the data is in a second portion of the cache 240 (operation 4003). The second cache block CB2 may have a second size that is smaller than the request size 12 and is different in size from (e.g., larger or smaller in size) the first size (operation 4003). Based on the read request 10, the storage device 200 may read the first portion of the data from the first cache block CB1, and may read the second portion of the data from the second cache block CB2 (operation 4004).

[0081] Thus, aspects of some embodiments of the present disclosure may provide improvements to computer storage systems by providing adaptive cache block sizes (or variable sized cache block allocations) to handle data requests in a manner that may reduce cache pollution and bandwidth pollution.

[0082] Example embodiments of the present disclosure may be extended to the following claims, but are not limited thereto:

[0083] Statement 1. An example method includes receiving, by a storage device, a read request for data, the read request being associated with a request size, determining that a first cache region associated with a first portion of the data is in a first portion of the cache, the first cache region having a first size that is less than the request size, determining that a second cache region associated with a second portion of the data is in a second portion of the cache, the second cache region having a second size that is less than the request size and different from the first size, and based on the read request, reading the first portion of the data from the first cache region and reading the second portion of the data from the second cache region.

[0084] Statement 2. An example method includes the method of statement 1, wherein the first portion of the cache includes a first cache group divided into cache areas having a first size, and the second portion of the cache includes a second cache group divided into cache areas having a second size.

[0085] Statement 3. An example method includes the method of any of statements 1 and 2, wherein determining that the first cache region is in a first portion of the cache comprises checking a first lookup data structure associated with the first size, and determining that the second cache region is in a second portion of the cache comprises checking a second lookup data structure associated with the second size.

[0086] Statement 4. An example method includes the method of statement 3, wherein the first lookup data structure includes a first key-value store and the second lookup data structure includes a second key-value store.

[0087] Statement 5. An example method includes the method of any of statements 1-4, wherein the first portion of the cache includes a first cache line and the second portion of the cache includes a second cache line, the first cache line being separate from the second cache line.

[0088] Statement 6. An example method includes the method of any of statements 1-5, wherein the first cache area and the second cache area are linked by a first least recently used (LRU) list.

[0089] Statement 7. An example method includes the method of any of statements 1-6, wherein the first LRU list comprises a group LRU list, and the first cache area and the second cache area are linked by a second LRU list different from the first LRU list.

[0090] Statement 8. An example apparatus for performing the method of any of statements 1-7, comprising a cache and processing circuitry.

[0091] Statement 9. An example system for performing the method of any of statements 1-7, comprising a processor and a memory storing instructions, the instructions causing the processor to perform the method of any of statements 1-7 upon execution by the processor.

[0092] While embodiments of the present disclosure have been particularly shown and described with reference to the embodiments described herein, it will be understood by those skilled in the art that various changes in form and details may be made without departing from the spirit and scope of the present disclosure as set forth in the following claims and their equivalents.

Claims

1. A method for data storage, the method comprising: receiving, by a storage device, a read request for data, the read request being associated with a request size; determining that a first cache region associated with a first portion of the data is in a first portion of a cache, the first cache region having a first size that is less than the request size; determining a second cache region associated with a second portion of the data in a second portion of the cache, the second cache region having a second size that is smaller than the request size and different from the first size; as well as Based on the read request, the first portion of the data is read from the first cache area, and the second portion of the data is read from the second cache area.

2. The method according to claim 1, wherein: The first portion of the cache includes a first cache group divided into cache regions having the first size; and The second portion of the cache includes a second cache group divided into cache regions having the second size.

3. The method according to claim 1, wherein: The determining that the first cache region is in the first portion of the cache includes examining a first lookup data structure associated with the first size; and The determining that the second cache region is in the second portion of the cache includes checking a second lookup data structure associated with the second size.

4. The method according to claim 3, wherein: The first lookup data structure includes a first key-value store, and the second lookup data structure includes a second key-value store.

5. The method according to claim 1, wherein: The first portion of the cache includes a first cache line and the second portion of the cache includes a second cache line, the first cache line being separate from the second cache line.

6. The method according to claim 1, wherein: The first cache area and the second cache area are linked by a first least recently used LRU list.

7. The method according to claim 6, wherein: The first LRU list comprises a group LRU list; and The first cache area and the second cache area are linked by a second LRU list different from the first LRU list.

8. A storage device, comprising: Cache; as well as Processing circuit, Wherein, the processing circuit is configured as follows: receiving a read request for data, the read request being associated with a request size; determining a first cache region associated with a first portion of the data in a first portion of the cache, the first cache region having a first size that is less than the request size; determining that a second cache region associated with a second portion of the data is in a second portion of the cache, the second cache region having a second size that is smaller than the request size and different from the first size; and Based on the read request, the first portion of the data is read from the first cache area, and the second portion of the data is read from the second cache area.

9. The storage device according to claim 8, wherein: The first portion of the cache includes a first cache group divided into cache regions having the first size; and The second portion of the cache includes a second cache group divided into cache regions having the second size.

10. The storage device according to claim 8, wherein: The processing circuit is configured to: determining that the first cache region is in the first portion of the cache by examining a first lookup data structure associated with the first size; as well as The second cache region is determined to be in the second portion of the cache by examining a second lookup data structure associated with the second size.

11. The storage device according to claim 10, wherein: The first lookup data structure includes a first key-value store, and the second lookup data structure includes a second key-value store.

12. The storage device according to claim 8, wherein: The first portion of the cache includes a first cache line and the second portion of the cache includes a second cache line, the first cache line being separate from the second cache line.

13. The storage device according to claim 8, wherein: The first cache area and the second cache area are linked by a first least recently used LRU list.

14. The storage device according to claim 13, wherein: The first LRU list comprises a group LRU list; and The first cache area and the second cache area are linked by a second LRU list different from the first LRU list.

15. A system for data storage, the system comprising: processor; as well as A memory storing instructions, upon execution by the processor, the instructions causing the processor to execute: receiving a read request for data, the read request being associated with a request size; determining that a first cache region associated with a first portion of the data is in a first portion of a cache, the first cache region having a first size that is less than the request size; determining a second cache region associated with a second portion of the data in a second portion of the cache, the second cache region having a second size that is smaller than the request size and different from the first size; as well as Based on the read request, the first portion of the data is read from the first cache area, and the second portion of the data is read from the second cache area.

16. The system of claim 15, wherein: The first portion of the cache includes a first cache group divided into cache regions having the first size; and The second portion of the cache includes a second cache group divided into cache regions having the second size.

17. The system of claim 15, wherein: The determining that the first cache region is in the first portion of the cache includes examining a first lookup data structure associated with the first size; and The determining that the second cache region is in the second portion of the cache includes checking a second lookup data structure associated with the second size.

18. The system of claim 17, wherein: The first lookup data structure includes a first key-value store, and the second lookup data structure includes a second key-value store.

19. The system of claim 15, wherein: The first portion of the cache includes a first cache line and the second portion of the cache includes a second cache line, the first cache line being separate from the second cache line.

20. The system of claim 15, wherein: The first cache area and the second cache area are linked by a first least recently used LRU list and by a second LRU list different from the first LRU list.

Images