System and method for collaborative caching with near storage memory

By using a scalable interval tree and analysis engine to optimize data access paths, the problem of data access latency between multi-level storage systems in computer storage systems is solved, achieving faster data access and processing efficiency.

CN119025442APending Publication Date: 2024-11-26SAMSUNG ELECTRONICS CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410665317.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2024-05-17
Filing Date
2024-05-27
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The problem of increased data access latency in computer storage systems, especially due to inadequate latency management caused by the complexity of data transfer between multi-tiered storage systems.

Method used

It employs data structures such as scalable interval trees to track the location of data in different memory hierarchies, reduces latency from multiple accesses by directly accessing data storage locations, supports multi-threaded access, and utilizes an analytics engine to optimize request allocation.

Benefits of technology

It improves data access speed, reduces access latency between storage system hierarchies, and enhances system response time and processing efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119025442A_ABST
    Figure CN119025442A_ABST
Patent Text Reader

Abstract

A system is disclosed. The system may include a processor, a first memory connected to the processor, and a second memory connected to the processor. The data structure may include entries that may identify that data is stored in a location. The location may include one of the first memory or the second memory.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Patent Application Serial No. 63 / 601,197 filed on November 20, 2023 and U.S. Provisional Patent Application Serial No. 63 / 469,364 filed on May 26, 2023 and U.S. Patent Application Serial No. 18 / 668,063 filed on May 17, 2024, all of which are incorporated herein by reference for all purposes. Technical Field

[0003] The present disclosure relates generally to memory and storage devices, and more particularly to improving response time for accessing memory. Background Art

[0004] Storage systems in computers continue to evolve and become more complex. As a result, the complexity of managing data as it transitions between tiers in the storage system also increases, and incorrect management of where data is currently stored can result in delays in returning the data to the application.

[0005] There is still a need to support faster data access. Summary of the invention

[0006] The machine may include a data structure that can track where various data elements are currently stored. The machine may use the data structure to speed up access to specific data elements.

[0007] An embodiment of the present disclosure provides a system for collaborative caching, comprising: a processor; a first memory connected to the processor; a second memory connected to the processor; and a data structure, the data structure comprising at least an entry, the entry identifying data being stored in a location, the location comprising one of the first memory or the second memory.

[0008] Another embodiment of the present disclosure provides a method for collaborative caching, comprising: receiving a data access request for data from an application running on a processor; identifying a data structure based on the data access request; identifying an entry in the data structure based on the data access request; identifying a location where the data is stored based on the entry in the data structure; and accessing the data from the location, wherein the location includes a first memory or a second memory.

[0009] Another embodiment of the present disclosure provides a method for collaborative caching, comprising: receiving a processing request from an application running on a first processor, the processing request to be applied to data; performing analysis on the processing request to determine a target for executing the processing request, the target comprising a first processor or a second processor associated with a device; and dispatching the processing request to the target. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings described below are examples of how embodiments of the present disclosure may be implemented and are not intended to limit the embodiments of the present disclosure. Various embodiments of the present disclosure may include elements not shown in a particular drawing and / or may omit elements shown in a particular drawing. The drawings are intended to provide illustrations and may not be to scale.

[0011] Figure 1 A machine including a storage device for accessing data from various memories according to an embodiment of the present disclosure is shown.

[0012] Figure 2 The embodiment according to the present disclosure is shown Figure 1 Details of the machine.

[0013] Figure 3 The embodiment according to the present disclosure is shown Figure 1 How a machine can access data from various types of storage.

[0014] Figure 4A It is shown that the embodiment according to the present disclosure can be Figure 1 The storage device associated with Figure 1 A first example arrangement of an accelerator.

[0015] Figure 4B It is shown that the embodiment according to the present disclosure can be Figure 1 The storage device associated with Figure 1 A second example arrangement of an accelerator.

[0016] Figure 4C It is shown that the embodiment according to the present disclosure can be Figure 1 The storage device associated with Figure 1 A third example arrangement of an accelerator.

[0017] Figure 4D It is shown that the embodiment according to the present disclosure can be Figure 1 The storage device associated with Figure 1 A fourth example arrangement of an accelerator.

[0018] Figure 5 The embodiment according to the present disclosure is shown Figure 1 Details of the storage device.

[0019] Fig. 6A FIG. 1 shows a method for managing locations where data is stored in various memories according to an embodiment of the present disclosure. Figure 3 First example of a data structure.

[0020] Figure 6B FIG. 1 shows a method for managing locations where data is stored in various memories according to an embodiment of the present disclosure. Figure 3 A second example of a data structure.

[0021] Figure 7 The embodiment according to the present disclosure is shown Figure 3 Details of the analysis engine.

[0022] Figure 8 The embodiment according to the present disclosure is shown Figure 7 Details of the calculator.

[0023] Fig. 9 The present invention is shown in the embodiment of the present invention. Figure 1 Machine processing Figure 3 Flowchart of an example process for requesting data access.

[0024] Fig.10 The present invention is shown in the embodiment of the present invention. Figure 1 Machine receiving Figure 3 Flowchart of an example process for requesting data access.

[0025] Fig.11 The present invention is shown in the embodiment of the present invention. Figure 1 A flowchart of an example process for a machine to lock and unlock access to data in various memories.

[0026] Fig.12 The embodiment according to the present disclosure is shown Figure 1 Machine processing Figure 3 A flowchart of an example process for processing a request.

[0027] Fig.13 The present invention is shown in the embodiment of the present invention. Figure 1 Machine receiving Figure 3 A flowchart of an example process for processing a request.

[0028] Fig.14A The present invention is shown in the embodiment of the present invention. Figure 1 The machine determines where to process Figure 3 A flowchart of an example process for processing a request.

[0029] Fig. 14B Continuing with the embodiment of the present disclosure Figure 1 The machine determines where to process Figure 3 Example of a process for handling requests Fig.14A Flowchart of the process. DETAILED DESCRIPTION

[0030] Reference will now be made in detail to embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. In the following detailed description, many specific details are set forth to enable a thorough understanding of the present disclosure. However, it should be understood that one of ordinary skill in the art may practice the present disclosure without these specific details. In other cases, well-known methods, processes, components, circuits, and networks are not described in detail to avoid unnecessarily obscuring aspects of the embodiments.

[0031] It should be understood that although the terms first, second, etc. can be used to describe various elements in this article, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the scope of the present disclosure, the first module can be referred to as the second module, and similarly, the second module can be referred to as the first module.

[0032] The terms used in the description of the present disclosure are only used for the purpose of describing specific embodiments and are not intended to limit the present disclosure. As used in the description of the present disclosure and the appended claims, the singular forms "one", "an" and "the" are intended to also include plural forms, unless the context clearly states otherwise. It will also be understood that the term "and / or" as used herein refers to and covers any and all possible combinations of one or more associated listed items. It will be further understood that when used in this specification, the terms "include" and / or "comprise" specify the presence of the features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or their groups. The components and features of the accompanying drawings are not necessarily drawn to scale.

[0033] Originally, computers included only two tiers of storage: main memory and tape drives or hard disk drives (storage devices). Applications could keep track of what data was currently in main memory or not, and therefore could access data directly from the main memory of the storage device.

[0034] But as computers have grown in complexity, so have the storage systems they use. Host processors can now include multiple layers of cache memory that can serve as faster access points for frequently used data, even faster than main memory. Storage devices have also evolved to include local memory. Solid-state drives (SSDs), which are even faster than hard disk drives, can benefit from the use of memory such as dynamic random access memory (DRAM), which can still be faster than flash memory. Thus, an SSD can use DRAM or other memory local to the device as a cache for data from flash memory, much like releasing a lock on an entry in a data structure for use by a thread can cache data stored in main memory.

[0035] Some SSDs even support direct access by the processor and applications to the local memory of the device. For example, cache coherent interconnect protocols such as Compute Express A protocol that allows a processor to access data from both device local memory and flash memory. (Compute Express Link and CXL are registered trademarks of the Compute Express Link Consortium in the United States.) Therefore, device local memory can act as a substitute for main memory in terms of processor accessibility.

[0036] In summary, it is not uncommon for a computer system to have six or more different layers where data can be stored: for example, three layers of processor cache, main memory, device local memory, and device persistent storage. There can also be multiple instances of an element at a particular layer: for example, if a computer system has three SSDs, each with flash memory and device local memory, there are three device local memories and three device persistent storage devices where data can be stored.

[0037] The conventional paradigm for accessing data is to query the processor cache to see if the data exists there. If not, main memory may be checked to see if it stores the data, then device local memory, and finally device persistent storage. (The latter two checks may be performed by the device's controller rather than by the processor, but the accesses to the tiers in that order are effectively the same.) If the data is ultimately found only in device persistent storage, the checks on the data in processor cache, main memory, and device local memory add latency to the return of the data: a latency that, while not necessarily large on a per-access basis, can become significant when multiplied by the number of data accesses that the processor may perform over time.

[0038] Embodiments of the present disclosure address this problem by providing different paradigms for accessing data. A data structure such as a scalable interval tree can be used to track where portions of a particular file can be stored. Each node in the data structure can specify the location of a given portion of a file. Data access requests by applications can be intercepted. Data structures can be identified and nodes can be located based on the data being requested. Nodes can indicate where the data is currently stored: processor cache, main memory, device cache, or persistent storage. The data can then be accessed directly from where it is actually stored and returned to the application, avoiding multiple data accesses to various layers of the storage system.

[0039] If data is stored in multiple locations—for example, in both device local memory and device persistent storage—the data may be accessed from the faster location (in this example, device local memory). In some embodiments of the disclosure, data may be stored in multiple locations not counting device persistent storage; in other embodiments of the disclosure, data may be stored in only one location other than device persistent storage. Thus, if the data is stored in a processor cache after access, the data may be removed from device local memory.

[0040] Each node in the data structure can also act as a lock on the data represented by that node. Even in the same data structure, different nodes can be locked by different threads. Therefore, the data structure can support multi-threaded access. The number of threads that can access the data structure simultaneously is theoretically unbounded (although in practice, the number of threads that the processor can support can be used as an upper limit on the number of threads that can access the data structure simultaneously).

[0041] The data structure itself can be relatively small: perhaps 0.0005% of the original file size. This means that a one-terabyte (TB) file can be represented by a data structure that is only about 32 megabytes (MB) in size.

[0042] The data structure may be updated as data is added to or evicted from various tiers in the storage system. Thus, when a processor stores data or evicts data from a processor cache, the processor may notify the data structure so that the data structure may be updated. When data is added to or evicted from a device cache, the device may similarly notify the data structure.

[0043] Embodiments of the present disclosure may also support the management of computing processing resources. If the device supports computing processing—for example, the device includes an accelerator or other processor—the function may check the data structure to see where the data currently resides. If the data is completely in the processor cache, the processor may perform the requested processing. If the data is completely on the device and the device is capable of performing the processing, the processing may be transferred to the device. And if the data is split between processor caches, the system may perform an analysis to determine whether the host processor, the device processor, or a combination of the two is more efficient in performing the processing.

[0044] Figure 1 A machine designed to access data from various memories according to an embodiment of the present disclosure is shown. Figure 1 In , machine 105 —which may also be referred to as a host or system—may include a processor 110 , a memory 115 , and a storage device 120 .

[0045] Processor 110 may be any type of processor. Figure 1 A single processor 110 is shown, but machine 105 may include any number of processors, each of which may be a single-core or multi-core processor, each of which may implement a reduced instruction set computer (RISC) architecture or a complex instruction set computer (CISC) architecture (among other possibilities), and which may be mixed in any desired combination.

[0046] The processor 110 may be coupled to a memory 115. The memory 115 (which may also be referred to as main memory) may be any kind of memory, such as flash memory, dynamic random access memory (DRAM), static random access memory (SRAM), persistent random access memory, ferroelectric random access memory (FRAM), or non-volatile random access memory (NVRAM), such as magnetoresistive random access memory (MRAM), etc. The memory 115 may also be any desired combination of different memory types and may be managed by a memory controller 125. The memory 115 may be used to store what may be referred to as "short-term" data: that is, data that is not expected to be stored for an extended period of time. Examples of short-term data may include temporary files, data used locally by an application (which may have been copied from other storage locations), etc.

[0047] The processor 110 and the memory 115 may also support an operating system under which various applications may run. These applications may issue requests (which may also be referred to as commands) to read data from or write data to the memory 115 or storage device 120. The storage device 120 may be accessed using a device driver 130.

[0048] The storage device 120 may be associated with an accelerator 135, which may also be referred to as a computational storage device, a computational storage unit, a computational storage device, or a computational device. Figures 4A-4D As discussed, the storage device 120 and the accelerator 135 may be designed and manufactured as a single integrated unit, or the accelerator 135 may be separate from the storage device 120. The phrase "associated with" is intended to cover a single integrated unit that includes both the storage device and the accelerator, as well as a storage device that is paired with the accelerator but is not manufactured as a single integrated unit. In other words, when the storage device and the accelerator are physically separate devices but are connected in a manner that enables them to communicate with each other, they can be said to be "paired." In addition, in the remainder of this document, any reference to the storage device 120 and / or the accelerator 135 may be understood to refer to the device as being physically separated but paired (and therefore may include another device), or integrating the two devices into a single component as a computing storage unit.

[0049] Additionally, the connection between the storage device and the paired accelerator may enable the two devices to communicate, but may not enable one (or both) devices to work with different partners: that is, the storage device may not be able to communicate with another accelerator, and / or the accelerator may not be able to communicate with another storage device. For example, the storage device and the paired accelerator may be connected to the structure serially (in either order) so that the accelerator can access information from the storage device in a way that the other accelerator may not be able to achieve. Note that accelerator 135 is optional and may be omitted in some embodiments of the present disclosure.

[0050] Although Figure 1 The general term "storage device" is used, but embodiments of the present disclosure may include any storage device format that may be associated with computing storage, examples of which may include hard disk drives and solid-state drives (SSDs). Any reference below to a specific type of storage device (e.g., "SSD") should be understood to encompass such other embodiments of the present invention.

[0051] Processor 105, storage device 120, and accelerator 135 are shown connected to fabric 140. Fabric 140 is intended to represent any structure along which information can be passed. Fabric 140 may include fabric that may be internal to machine 105 and that may use an interface such as Peripheral Component Interconnect Express (PCIe), Serial AT Attachment (SATA), or Small Computer System Interface (SCSI). Fabric 140 may also include fabric that may be external to machine 105 and that may use an interface such as Ethernet, Infiniband, or Fibre Channel. In addition, fabric 140 may support one or more protocols such as Non-Volatile Memory Express (NVMe), NVMe over Fabric (NVMe-oF), Simple Service Discovery Protocol (SSDP), or a cache coherent interconnect protocol such as Compute Express. (Compute Express Link and CXL are registered trademarks of the Compute Express Link Consortium in the United States.) Therefore, fabric 140 can be considered to include both internal and external network connections, through which commands can be sent directly or indirectly to storage device 120 (and more specifically, accelerator 135 associated with storage device 120). In embodiments of the present disclosure where fabric 140 supports external network connections, storage device 120 and / or accelerator 135 can be located external to machine 105.

[0052] Figure 1The processor 105, storage device 120, and accelerator 135 are shown as connected to the structure 140 because the processor 105, storage device 120, and accelerator 135 can communicate via the structure 140. In some embodiments of the present disclosure, the storage device 120 and / or accelerator 135 may include a connection to the structure 120, which may include the ability to communicate with a remote machine and / or network: for example, a network-enabled solid-state drive (SSD). However, in other embodiments of the present disclosure, although the machine 105 may include a connection to another machine and / or network (which connection may be considered part of the structure 140), the storage device 120 and / or accelerator 135 may not be connected to another machine and / or network. In such an embodiment of the present disclosure, the storage device 120 and / or accelerator 135 may still be reached from the remote machine, but such commands may reach the storage device 120 and / or accelerator 135 through the processor 110, among other possibilities.

[0053] Figure 2 The embodiment according to the present disclosure is shown Figure 1 Details of the machine. Figure 2 In the embodiment of the present invention, generally, the machine 105 includes one or more processors 110, which may include a memory controller 125 and a clock 205, which may be used to coordinate the operation of the components of the machine. The processor 110 may also be coupled to a memory 115, which may include, by way of example, a random access memory (RAM), a read-only memory (ROM), or other state-retaining medium. The processor 110 may also be coupled to a storage device 120, and to a network connector 210, which may be, for example, an Ethernet connector or a wireless connector. The processor 110 may also be connected to a bus 215, to which a user interface 220 and input / output (I / O) interface ports that may be managed using an I / O engine 225, as well as other components, may be attached.

[0054] Figure 3 The embodiment according to the present disclosure is shown Figure 1 How the machine 105 can access data from various memories. Figure 3 middle, Figure 1 The machine 105 may include various different memories. These may include, for example, a processor software cache 305 (which may be in Figure 1 cache within the processor 110 or within the memory 115), the memory 115, and the storage device 120.

[0055] The processor software cache 305 may include Figure 1 The processor software cache 305 may represent a cache that may be included in the processor 110. Figure 1Any number of processor caches in processor 110. For example, Figure 1 The processor 110 may include L1, L2, and L3 caches, each of which may provide different responsiveness and capacity: Figure 1 Data may be moved between the various levels of the processor software cache 305 as deemed appropriate by the processor 110. Note that in some embodiments of the present disclosure, the processor software cache 305 may be Figure 1 External to processor 110 : for example, a DRAM cache in or near memory 115 .

[0056] The storage device 120 itself may include a device local memory 310 (which may also be referred to as device memory or device cache) and a device persistent storage 315 (which may be referred to as device storage). In some embodiments of the present disclosure, the device local memory 310 may include flash memory, DRAM, SRAM, persistent random access memory, FRAM, or NVRAM (such as MRAM), and the device persistent storage 315 may include a disk that may be found in a hard drive or flash memory that may be found in an SSD. The device local memory 310 may be a variety of volatile memories, and the device persistent storage 315 may be a variety of non-volatile memories.

[0057] In some embodiments of the present disclosure, the device local memory 310 may act as a cache for data that is otherwise stored in the device persistent storage 315. That is, when data is read from the storage device 120, the data may first be loaded from the device persistent storage 315 into the device local memory 310 and then back. Similarly, when data is written to the storage device 120, the data may first be written to the device local memory 310 and then later transferred to the device persistent storage 315 for more permanent (persistent) storage.

[0058] In some embodiments of the present disclosure, storage device 120 may expose an address range consistent with device persistent storage 315, such as a storage device (accessible by reading or writing pages, blocks, or sectors, depending on the implementation of device persistent storage 315). In such embodiments of the present disclosure, Figure 1The processor 110 may use an input / output (I / O) queue or submission queue to issue data access requests, such as read requests or write requests, to the storage device 120. The storage device 120 may then access the data access requests from such a queue and may return the results in an appropriate manner: for example, via a completion queue. In other embodiments of the present disclosure, the storage device 120 may expose an address range as if the storage device 120 were another form of memory similar to the memory 115: in these embodiments of the present disclosure, the address range exposed by the storage device 120 may be the same size as or smaller than the storage capacity provided by the device persistent storage 315. In such embodiments of the present disclosure, the device local memory 310 may be Figure 1 The processor 110 is accessible, much like the memory 115 may be Figure 1 In other embodiments, the storage device 120 may support Figure 1 The processor 110 of the device accesses the device local memory 310 similar to how the memory 115 may be accessed, and reads data from and writes data to the device persistent storage 315 , such as a storage device.

[0059] In some embodiments of the present disclosure, data may reside in both the processor software cache 305 (or memory 115) and the device local memory 310. However, in other embodiments of the present disclosure, data stored in the processor software cache 305 (or memory 115) should not be stored in the device local memory 310, and vice versa. That is, data may be stored exclusively in the device local memory 310 or in the processor software cache 305 / memory 115, but not both.

[0060] In response to a read or write request, such as data access request 320, Figure 1The processor 110 may sequentially search the various memories 305, 115, 310, and 315 to try to locate the data. For example, the fastest memory, such as the processor software cache 305, may be searched first; if the data is not located in the processor software cache 305, the next fastest memory (such as a DRAM cache) may be searched, then the memory 115 may be searched, then the device local memory 310 may be searched, and finally if the data is not found anywhere else, the device persistent storage 315 may be searched. However, by sequentially searching the memories 305, 115, 310, and 315, the time required to access the data from each memory 115, 310, and 315 becomes cumulative relative to the time required to search all earlier memories. For example, if the time to access the L1, L2, and L3 processor caches 305 is 2 nanoseconds (ns), 10ns, and 20ns, respectively, the time to access memory 115 is 60ns, the time to access device local memory 310 is 400ns (device local memory 310 may use the same DRAM as memory 115, but is on a different bus which may increase latency, as well as some additional time for the controller of storage device 120 to process the request), and by the time memory 115 searches for data—50 microseconds (μs)—32ns have passed (2+12+20 for each level of processor software cache 305), which means that 32ns have passed before memory 115 is even accessed to try to locate the requested data. This increased time is passed to each memory later in sequence.

[0061] Embodiments of the present disclosure avoid this increased latency by using a data structure 325. The data structure may specify where the requested data is currently stored: in the processor software cache 305, in the memory 115, in the device local memory 310, or in the device persistent storage 315. The data access request can then be sent directly to the memory where the data is stored, without incurring the additional latency of searching other memories. Using the data structure 325 may involve a small delay to determine where the data is actually located (or, for data stored in multiple locations, to determine all locations where the data is stored), but the query may be less than the time required to search each memory individually and sequentially. Although the use of the data structure 325 may result in slightly slower accesses from the processor software cache 305 (or the first memory in the sequence), typically the first memory in the sequence is the memory with the smallest capacity, and therefore may only have an infrequent negative impact on data access time: most data accesses may be faster. Refer below to Figure 6A-6B Data structure 325 is discussed further.

[0062] In some embodiments of the present disclosure, Figure 1The processor 110 of the present disclosure may support multi-threaded execution (such as multiple threads of an application 330, or threads of multiple applications 330). In such an embodiment of the present disclosure, the machine 105 may be able to support multi-threaded access to the data structure 325. Because multi-threaded access may create potential conflicts for different threads attempting to access the same data at the same time, the data structure 325 may support locking various portions of the data. Figure 6A-6B Further discussion of data locking. In theory, the number of threads that can access data structure 325 is unbounded (except for Figure 1 10), but in practice there may be an upper limit (such as 100-200) to the number of threads that can simultaneously access data structure 325 before access to data structure 325 may become relatively slow and become a bottleneck in itself.

[0063] To use data structure 325, Figure 1 The machine 105 may implement a library 335. The library 335 may be a library of routines that support the use of the data structure 325. For example, the library 335 may include functionality to intercept data access requests. That is, when the application 330 issues a data access request 320, the library 335 may intercept the request before it reaches regular processing. The library 335 may then determine what data is being accessed in the data access request 320, use the data structure 325 to determine where the data is currently stored, and then direct the data access request 320 directly to the appropriate memory. For example, if the accessed data can be found in the device local memory 310, the library 335 may direct the data access request 320 to the device local memory 310, bypassing any search of the processor software cache 305 or memory 115.

[0064] In order for data structure 325 to be useful, data structure 325 should be updated whenever data is added or evicted from memory. Thus, when data is added to or evicted from processor software cache 305, memory 115, or storage device 120 (most typically in device local memory 310, but in some embodiments of the present disclosure, changes to data in device persistent storage 315 may also be reflected in data structure 325), data structure 325 may be updated accordingly. This fact is represented by dashed lines 340, 345, and 350. Figure 1 The processor 110 (or more precisely, the Figure 1The kernel of an operating system running on processor 110 of the storage device 120 may notify library 335 of changes to the data stored therein so that the functions of library 335 may update data structure 325. Similarly, storage device 120 (more specifically, the controller of storage device 120) may notify library 335 of changes regarding what data is stored where so that the functions of library 335 may update data structure 325.

[0065] Application 330 may also issue a request to process data using a specific function, which may be known in advance: a standard function provided by a library (which may be different from library 335) or a custom designed function. Figure 1 The processor 110 may be capable of performing the function on the identified data, but in some embodiments of the present disclosure, the storage device 120 may include a processor 355, which may be Figure 1 The accelerator 135 version (will Figure 3 and Figure 1 For comparison, Figure 3 A single computing storage unit including both memories 310 and 315 and a processor 355 is shown, Figure 1 Shows Figure 1 storage device 120 and accelerator 135 as paired but separate elements: both arrangements are possible in various embodiments of the present disclosure). That is, processor 355 may provide Figure 1 The processor 110 of the present disclosure may be closer to some local processing capabilities of the data on the storage device 120, and the storage device 120 may be a computational storage unit. In such an embodiment of the present disclosure, it may be more efficient to process the data locally on the processor 355 rather than having the processor 110 perform the function. For example, if the data all resides on the device persistent storage 315 and is not available in the processor software cache 305 or memory 115, then using Figure 1 It may take more time for processor 110 to perform the function than for processor 355 to perform the function (e.g., in moving data from device persistent storage 315 to memory 115).

[0066] To determine where such a functional engine is located, embodiments of the present disclosure may include an analysis engine 360. The analysis engine 360 ​​may perform calculations to estimate where such a functional engine is located. Figure 1 The processor 110 of the processor 355 is relative to the time required to perform the function on the processor 355, and the processing request 365 is dispatched to the appropriate element to perform the requested function. The processing request 365 can be dispatched to Figure 1 The processor 110, the processor 355 or even both of them can perform functions in cooperation. Figure 7-8 Analysis engine 360 ​​is discussed further.

[0067] Figures 4A-4D It is shown that the embodiment according to the present disclosure can be Figure 1 The storage device 120 is associated with Figure 1 Various arrangements of the accelerator 135. Figure 4A , a storage device 405 (which may be Figure 1 storage device 120) and computing device 410-1 (which may be Figure 1 Accelerator 135 or Figure 3 The storage device 405 may include a controller 415 and a storage device 420-1. The storage device 405 may be accessible using any desired access format. For example, Figure 4A In , the storage device 405 can be accessed across the fabric 140 using a submission queue and a completion queue, which can form a queue pair. Figure 4A , storage device 405 is shown to include two queue pairs: management queue pair 425 can be used to manage storage device 405; and I / O queue pair 430 can be used to control the I / O of storage device 405. Embodiments of the present disclosure may include any number (one or more) of queue pairs 425 and 430 (or other forms of access), and access may be shared: for example, a single queue pair may be used for both management and I / O control of storage device 405 (i.e., queue pairs 425 and 430 may be combined in one queue pair).

[0068] The computing device 410-1 may be paired or associated with the storage device 405. The computing device 410-1 may include any number of (one or more) processors 435, which may also be referred to as computing storage processors, computing engines, or engines, and the processors 435 may provide one or more services 440-1 and 440-2, which may be collectively referred to as services 440, and which may also be referred to as computing storage services (CSS) or functions. For greater clarity, each processor 435 may provide any number of (one or more) services 440 (although embodiments of the present disclosure may include computing devices 410-1 that include, for example, Figure 4A The services 440 may be functions built into the processor 435, Figure 1 The processor 110 downloads the function (ie, Figure 1435 ) or both. Computing device 410-1 may be reachable across management queue pair 445 and / or I / O queue pair 450, which may be used to manage computing device 410-1 and / or control I / O of computing device 410-1, respectively, similar to queue pairs 425 and 430 used for storage device 405. (Management queue pair 445 and I / O queue pair 450 may be more generally referred to as queue pairs 445 and 450 without reference to their specific uses). Similar to queue pairs 425 and 430, other forms of access other than queue pairs 445 and 450 may be used, and a single queue pair may be used for both management and I / O control of computing device 410-1 (i.e., queue pairs 445 and 450 may be combined in one queue pair).

[0069] Processor 435 can be considered as near-storage processing: that is, Figure 1 The processor 110 is closer to the storage device 405. Because the processor 435 is closer to the storage device 405, Figure 1 Processor 435 is able to execute commands on data stored in storage device 405 faster than processor 110 can execute commands. Figure 4A 4, but the processor 435 may have associated memory that can be used to execute commands locally on data stored in the storage device 405.

[0070] Although Figure 4A Storage device 405 and computing device 410-1 are shown as being individually reachable across fabric 140, but embodiments of the present disclosure may also include storage device 405 and computing device 410-1 connected in series. That is, commands directed to storage device 405 and computing device 410-1 may both be received at the same physical connection to fabric 140 and may reach the other through one device. For example, if computing device 410-1 is located between storage device 405 and fabric 140, computing device 410-1 may receive commands directed to both computing device 410-1 and storage device 405: computing device 410-1 may process commands directed to computing device 410-1 and may pass commands directed to storage device 405 to storage device 405.

[0071] Service 440 can provide a plurality of different functions that can be performed on the data stored in storage device 405. For example, service 440 can provide predefined functions, such as encryption, decryption, compression and / or decompression of data, erasure coding, and / or application of regular expressions. Alternatively, service 440 can provide more general functions, such as data search and / or SQL functions. Service 440 can also support running application-specific code. That is, an application using service 440 can provide custom code to be executed using data on storage device 405. In some embodiments of the present disclosure, service 440 can be stored in a "program slot": that is, a specific address range within processor 435. Service 440 can also be any combination of these functions. Table 1 lists some examples of services that can be provided by processor 435.

[0072] Table 1: Service Types

[0073]

[0074]

[0075] Processor 435 (and indeed computing device 410-1) can be implemented in any desired manner. Example implementations may include local processors such as a central processing unit (CPU), a graphics processing unit (GPU), a general purpose GPU (GPGPU), a data processing unit (DPU), and a tensor processing unit (TPU), among other possibilities. Processor 435 may also be implemented using a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or a system on a chip, among other possibilities. If computing device 410-1 includes more than one processor 435, each processor 435 may be implemented as described above. For example, computing device 410-1 may have each of a CPU, a TPU, and an FPGA, or computing device 410-1 may have two FPGAs, or computing device 410-1 may have two CPUs and one ASIC, and so on.

[0076] Depending on the desired interpretation, computing device 410 - 1 , processor 435 , or a combination thereof may be considered a computational storage unit.

[0077] although Figure 4A Storage device 405 and computing device 410-1 are shown as separate devices, but Figure 4B Thus, computing device 410-2 may include controller 415, storage device 420-1, and processor(s) 435 that provide services 440-1 and 440-2. Figure 4AAs with the storage device 405 and the computing device 410-1, management and I / O commands may be received via the queue pairs 445 and / or 450. Even though the computing device 410-2 is shown as including both storage and the processor 435, Figure 4B It may still be considered to include storage devices associated with computing storage units.

[0078] exist Figure 4C In yet another variation, a computing device 410-3 is shown. The computing device 410-3 may include a controller 415 and a storage device 420-1, as well as a processor (or processors) 435 that provide services 440-1 and 440-2. However, even though the computing device 410-3 may be considered a single component including the controller 415, the storage device 420-1, and the processor 435 (and also considered a storage device associated with a computing storage unit), the computing device 410-3 may include a controller 415, a storage device 420-1, and a processor 435. Figure 4B , the controller 415 and the processor 435 may each include their own queue pairs 425 and / or 430 and 445 and / or 450 (which may also be used for management and / or I / O). By including the queue pairs 425 and / or 430, the controller 415 may provide transparent access to the storage device 420-1 (rather than requiring all communications to go through the processor(s) 435), while the queue pairs 445 and / or 450 may be used to access the processor(s) 435.

[0079] Additionally, the processor(s) may have proxy storage access 455 for accessing storage device 420 - 1 . Instead of routing access requests through controller 415 , processor(s) 435 may be able to directly access data from storage device 420 - 1 using proxy storage access 455 .

[0080] exist Figure 4C In FIG. 4 , the controller 415 and the proxy storage access 455 are both shown with dashed lines to indicate that they are optional elements and may be omitted depending on the implementation.

[0081] at last, Figure 4D Another embodiment is shown. Figure 4D , a computing device 410-4 is shown, which may include an array, which may include one or more storage devices 420-1 to 420-4. Figure 4D Four storage elements are shown, but embodiments of the present disclosure may include any number (one or more) of storage elements. In addition, the individual storage elements may be other storage devices, such as Figures 4A-4D Those storage devices shown in .

[0082] Because computing device 410-4 may include more than one storage element 420-1 to 420-4, computing device 410-4 may include array controller 460. Array controller 460 may manage how data is stored on and retrieved from storage elements 420-1 to 420-4. For example, if storage elements 420-1 to 420-4 are implemented as some level of a redundant array of independent disks (RAID), array controller 460 may be a RAID controller. If storage elements 420-1 to 420-4 are implemented using some form of erasure coding, array controller 460 may be an erasure code controller.

[0083] Figure 5 The embodiment according to the present invention is shown Figure 1 Details of the storage device 120. Figure 5 In , the storage device 120 is illustrated using an implementation including an SSD 120, but embodiments of the present disclosure are applicable to any type of storage device that can perform garbage collection or media management, as described below.

[0084] SSD 120 may include an interface 505 and a host interface layer 510. Interface 505 may be used to connect SSD 120 to a host. Figure 1 The SSD 120 may include more than one interface 505: for example, one interface may be used for block-based read and write requests, and another interface may be used for key-value read and write requests. Figure 5 The recommended interface 505 is Figure 1 The physical connection between SSD 120 and machine 105 is represented by interface 505, but interface 505 may also represent protocol differences that may be used across a common physical interface. For example, SSD 120 may connect to machine 105 using U.2, Enterprise and Data Center Standard Form Factor (EDSFF), or M.2 connectors, among other possibilities, and SSD 120 may support block-based requests and key-value requests: processing different types of requests may be performed by different interfaces 505. SSD 120 may also include a single interface 505 that may include multiple ports, each of which may be treated as a separate interface 505, or simply a single interface 505 with a single port, and leaving the interpretation of information received through interface 505 to another element, such as SSD controller 515.

[0085] Host interface layer 510 may manage interface 505, providing an interface between SSD controller 515 and external connections to SSD 120. If SSD 120 includes more than one interface 505, a single host interface layer 510 may manage all interfaces, SSD 120 may include a host interface layer 510 for each interface, or some combination thereof may be used.

[0086] SSD 120 may also include SSD controller 515 and various flash memory chips 520-1 to 520-8, which may be organized along channels 525-1 to 525-4. Flash memory chips 520-1 to 520-8 may be collectively referred to as flash memory chips 520, and may also be referred to as flash memory chips, memory chips, NAND chips, chips, or dies. Channels 525-1 to 525-4 may be collectively referred to as channels 525. Flash memory chips 520 may collectively represent Figure 3 The SSD controller 515 may manage sending read requests and write requests to the flash chips 520 along the channel 525. The controller 515 may also include a flash controller 530, which may be responsible for issuing commands to the flash chips 520 along the channel 525. In embodiments of the present disclosure where the storage device 120 uses a technology other than the flash chips 520 to store data, the flash controller 530 may also be more generally referred to as a memory controller 530. Although Figure 5 Eight flash die 520 and four channels 525 are shown, but embodiments of the present disclosure may include any number (one or more, unbounded) of channels 525 including any number (one or more, unbounded) of flash die 520 .

[0087] Within each flash memory chip or die, space can be organized into planes. These planes can include multiple erase blocks (also referred to as blocks), which can be further subdivided into word lines. A word line can include one or more pages. For example, a word line for a triple-level cell (TLC) flash memory medium can include three pages, while a word line for a multi-level cell (MLC) flash memory medium can include two pages.

[0088] The erase blocks may also be logically grouped together by the controller 515, which may be referred to as a super block. This logical grouping may enable the controller 515 to manage the group as one, rather than managing each block individually. For example, a super block may include one or more erase blocks from each plane of each die in the storage device 120. Thus, for example, if the storage device 120 includes eight channels, two dies per channel, and four planes per die, the super block may include 8×2×4=64 erase blocks.

[0089] SSD controller 515 may also include a flash translation layer (FTL) 535 (which may be more generally referred to as a translation layer for storage devices that do not use flash storage). FTL 535 may handle LBA or other logical IDs (such as those specified by Figure 1 FTL 535 may also be responsible for tracking data as it is relocated from one PBA to another, such as may occur when performing garbage collection and / or wear leveling.

[0090] Finally, in some embodiments of the present disclosure, SSD controller 515 may include device local memory 310 and processor 355 (in embodiments of the present disclosure where SSD 120 includes processor 355). Note that SSD 120 may include device local memory 310 and / or processor 355 elsewhere in SSD 120 besides SSD controller 515: Figure 5 SSD controller 515 is shown as including device local memory 520 and processor 355 merely as example locations for these elements.

[0091] Fig. 6A FIG. 1 shows a method for managing locations where data is stored in various memories according to an embodiment of the present invention. Figure 3 A first example of a data structure 325. Fig. 6A In the embodiment of the present invention, data structure 325 may include a tree, such as a scalable interval tree. An interval tree is a data structure used to search for information that may involve a range of values. For efficient searching, the interval tree may be balanced so that the worst case time for searching the interval tree is generally not significantly worse than the average case time for searching the interval tree.

[0092] Scalable interval tree 325 is shown to include nodes 605-1 through 605-5, which may be collectively referred to as nodes 605. Each node 605 may represent a partial range of data: for example, Figure 1 In some embodiments of the present disclosure, the scalable interval tree 325 (or data structure 325) may be associated with a single file rather than a single file. Figure 1 The entire data span of the storage device 120 is associated, in which case there may be Figure 1 The machine 105 uses more than one scalable interval tree 325 / data structure 325.

[0093] Each node 605 may identify data associated with the node 605. For example, node 605-1 may represent data associated with an address range of 6-8 megabytes (MB), while node 605-4 may represent data associated with an address range of 0-2 MB.

[0094] Each node 605 may also indicate where the data it represents is currently stored. For example, Fig. 6A The cross-hatching in the Figure 1 The data in the memory 115 and the diagonal line may represent the data stored in Figure 3 Thus, the data represented by nodes 605-1 and 605-5 may currently be stored in Figure 3 , while the data represented by nodes 605-2, 605-3, and 605-4 may currently be stored in the device local memory 310. Figure 1 Additional indicators may be used for other types of memory, such as Figure 3 Various levels of processor software cache 305 or Figure 3 Device persistent storage 315. In addition, node 605 may indicate that specific data is stored in multiple locations. For example, specific data may be stored in Figure 1 Memory 115, Figure 3 The device local memory 310 and Figure 3 The device persistent storage 315 can then be selected to provide the fastest access to the data taking into account all relevant considerations. These factors may include the time to access the relevant location, the time required by the host and / or device to complete the request, the bandwidth and queue length of the device, and the queue processing time. For example, the data currently stored in Figure 1 The data in the memory 115 may currently have a relatively long queue of requests waiting to be completed, while Figure 3 The device persistent storage 315 may have a relatively short request queue and receive requests from Figure 3 The device persistent storage device 315 may actually read data faster than reading from Figure 1 The memory 115 reads data faster

[0095] In some embodiments of the present disclosure, in particular, only Figure 3 In an embodiment of the present disclosure of a device persistent storage device 315, it can be assumed that data is always available from Figure 3 The data structure 325 can therefore be omitted from the list of locations where data can be stored. Figure 3 device persistent storage 315, and if not otherwise indicated in data structure 325, it may be assumed that Figure 3The device persistent storage 315 is the only available location for data.

[0096] Note the lock symbols 610-4 and 610-5 (which may be collectively referred to as locks 610) next to nodes 605-4 and 605-5, respectively. Figure 3 As mentioned, multithreaded access to data structure 325 may result in conflicts in attempts to access data. For example, if two different threads both attempt to write the same data, the data may remain in an inconsistent state, or one thread may believe that its write is still valid when in fact its data is overwritten by another thread. To avoid this, node 605 may support locks 610. Locks 610 may be issued to a particular thread, where another thread is blocked from accessing data until lock 610 is released. Locks 610 may be allocated and released using any desired lock method.

[0097] Because scalable interval tree 325 may include any number of nodes 605 in any configuration, nodes 605 may be implemented using a data structure that includes data and pointers to other locations in memory, which may support easy insertion and deletion of nodes 605 as data is updated.

[0098] Figure 6B FIG. 1 shows a method for managing locations where data is stored in various memories according to an embodiment of the present invention. Figure 3 A second example of a data structure 325. Rather than using a dynamically managed data structure like a tree with pointers, the data structure 325 can be implemented using an array of entries 605-1 to 605-5 that include related information. For example, in Figure 6B 605-1 to 605-5 (which may be collectively referred to as entry 605). The entries may include address ranges 615-1 to 615-5 (which may be collectively referred to as address ranges 615), locations 620-1 to 620-5 (which may be collectively referred to as locations 620), and a lock 610 indicating whether the data is currently locked. For example, entry 605-1 may represent data associated with address range 6-8MB (address range 615-1), which may currently be stored in Figure 3 620-1) and is currently unlocked (lock 610-1). On the other hand, entry 605-4 may represent data associated with address range 0-2MB (address range 615-4), which may currently be stored in Figure 3 620-4) and is currently locked (lock 610-4). For the currently locked entry 605, the entry 605 may also include the thread identifier that currently holds the lock on the data. Fig. 6A As discussed, table 325 may be modified to reflect that various data may be available at multiple locations and to indicate at which locations the data may be located.

[0099] Whether data structure 325 is implemented as Fig. 6A The scalable interval tree shown in Figure 6B Whether the table shown or any other implementation, all of which may be included in various embodiments of the present disclosure, data structure 325 may be relatively small. For example, the amount of data required to store a terabyte (TB) of memory may be only 32MB: approximately 0.0005% of the data size. Therefore, even when Figure 1 The machine 105 includes a very large number of Figure 1 The memory 115 and Figure 3 When the memory 305, 310 and 315 are connected, the data structure 325 should also be easily adapted to Figure 1 115. Note that the size of the data range managed by each entry 605 can be set to any desired value: a 2MB range is presented only as an example. Larger data range sizes can result in fewer entries 605 (or Fig. 6A 605), resulting in a smaller size of the data structure 325; a smaller data range size can result in more entries 605 (or Fig. 6A ), resulting in a larger size of data structure 325.

[0100] As reference Figure 3 As mentioned, when data is added to Figure 3 processor software cache 305, Figure 1 Memory 115, Figure 3 Device local storage 310 and / or Figure 3 device persistent storage device 315 or from Figure 3 processor software cache 305, Figure 1 Memory 115, Figure 3 Device local storage 310 and / or Figure 3 When the device persistent storage device 315 is evicted, Figure 1 The processor 110 and / or Figure 1 The storage device 120 may notify Figure 3 The functions of library 335 can then use this information to update accordingly Figure 6A-6B data structure 325 to maintain current information about what data is stored where.

[0101] Figure 7 The embodiment according to the present disclosure is shown Figure 3 Details of the analytical engine 360. Figure 7 In the example embodiment, the analysis engine 360 ​​may include a data locator 705. The data locator 705 is not intended to be Figure 3 Instead of the data structure 325, it is possible to determine the Figure 3 What data is processed by 365 and can it be used Figure 3 The data structure 325 determines where the data is currently stored. Then, the calculator 710 can calculate Figure 1 The processor 110 executes the processing request and Figure 3 The estimated time for the processor 355 to execute the processing request. Then, if Figure 1 The processor 110 and Figure 3 The processors 355 are to work cooperatively to execute Figure 3 The processing request 365, the allocation unit 715 can Figure 3 Requests for processing 365 are assigned to Figure 1 processor 110, Figure 3 processor 355 or both.

[0102] Note that in some cases, the allocation unit 715 should allocate Figure 3 The choice of handling request 365 may be obvious. For example, if Figure 1 The storage device 120 does not include the processor 355 of FIG. 4, or if Figure 3 The processor 355 is only capable of executing Figure 3 If the pre-installed function of the function requested in the processing request 365 is Figure 3 Request 365 is dispatched to Figure 1 Similarly, if the processor 355 of FIG. 4 is capable of executing Figure 3 365, and all data to be processed is currently in Figure 3 Device local storage 310 or Figure 3 In the device persistent storage device 315, Figure 3 The processor 355 executes Figure 3 The processing of 365 requests may be faster than Figure 1 The processor 110 executes Figure 3 The processing of 365 requests is more efficient (although Figure 3 The processing backlog of processor 355 may change this result). However, in the more general case, the data may be Figure 1 The memory 115 and Figure 3 The analysis becomes more complicated by splitting between the device local memory 310 and using the result of the calculator 710.

[0103] Figure 8 The embodiment according to the present disclosure is shown Figure 7 Calculator 710 may use various data to calculate estimated host processing time 805 and estimated device processing time 810, and then Figure 7 The allocation unit 715 can use this data to appropriately dispatch Figure 3 Such calculations may take into account where the data to be processed is currently stored (e.g., as used by data locator 705). Figure 3 ), host execution time 820, and device execution time 825 (which assumes all data is available, estimated Figure 1 The processor 110 and Figure 3 The processor 355 may each take how long to execute Figure 3 of processing requests 365), bandwidth 830 (which can be considered Figure 3 processor software cache memory 305, Figure 1 The memory 115 and Figure 3 how long it takes to move data between the device local memory 310), I / O queue length 835 (which can indicate the Figure 1 How many requests are already pending in the I / O queue of the memory device 120) and the I / O queue processing time 840 (which may specify how long it takes on average for the storage device 120 to handle a single I / O request in the I / O queue).

[0104] Generally, the host processing time 805 and the device processing time 810 can be estimated using the following equations:

[0105]

[0106]

[0107] In the above equation, T refers to the estimated time, DR refers to the data rate in a specific memory, B refers to the bandwidth between memories, E refers to the average execution time, and C davg refer to Figure 1 The average time for the storage device 120 to process an I / O request, and Q len refer to Figure 1 The length of the I / O queue of the storage device 120. The subscript h refers to Figure 1 The processor 110, d refers to Figure 3 Processor 355, hm refers to Figure 1 Processor software cache 305 / Figure 1 The memory 115, dm refers to Figure 3 The device local memory 310, and ds refers to Figure 3The device persistent storage device 315. For bandwidth, two subscripts are shown to indicate the bandwidth between two devices, because different pairs of devices can have different bandwidths between them. So, for example, B hm-dm refer to Figure 1 The memory 115 and Figure 3 The bandwidth between the device local memory 310, and B ds-dm refer to Figure 3 The device persistent storage device 315 and Figure 3 The bandwidth between the device local storage 310.

[0108] Calculator 710 can take the various inputs shown and generate Figure 1 The processor 110 and Figure 3 The estimated time for processor 355 to execute processing request 365. Note that other embodiments of the present disclosure may apply different formulas to estimate host and device processing times, and may use different combinations of inputs (and even Figure 8 different inputs not shown at all in the example).

[0109] In some embodiments of the present disclosure, hard-coded rules may be used. That is, assuming that data is stored in a particular location, requests to access the data may always be sent to that location, regardless of other factors. Thus, for example, if it is known that the data in question is stored in Figure 3 PSC 305, then any request for access to data may be automatically sent to Figure 3 PSC 305, even though data may also be stored in other layers. Similarly, hard-coded rules may be used to indicate that a particular request to process data may be sent to a particular device. For example, embodiments of the present disclosure may direct a particular processing request 365 to a Figure 1 of the particular processor 110, regardless of Figure 1 The availability of other processors 110 is determined.

[0110] Fig. 9 The present invention is shown in the embodiment of the present invention. Figure 1 105 machines are processed Figure 3 A flowchart of an example process for a data access request 320. Fig. 9 In block 905, Figure 3 The library 335 can be obtained from Figure 3 Application 330 received Figure 3 320. At block 910, the Figure 3 Data structure 325. For example, it can be based on Figure 3 The data access request 320 requests what data to identify Figure 3 At block 915, it is possible to identify Figure 3 6 of the data structure 325. At block 920, based on Figure 3 The entry 605 of FIG. 6 in the data structure 325 may identify Figure 1 Memory 115, Figure 3 processor software cache 305, device local memory 310 or Figure 3 Finally, at block 925, the Figure 1 Memory 115, Figure 3 processor software cache 305, device local memory 310 or Figure 3 Device persistent storage 315 access Figure 3 The data requested in the data access request 320.

[0111] Fig.10 The present invention is shown in the embodiment of the present invention. Figure 1 Machine 105 receives Figure 3 A flowchart of an example process for a data access request 320. Fig.10 In block 1005, Figure 3 Library 335 can intercept Figure 3 Application of 330 Figure 3 Data access request 320.

[0112] Fig.11 The present invention is shown in the embodiment of the present invention. Figure 1 A flowchart of an example process for locking and unlocking access to data in various memories by the machine 105. Fig.11 6, at block 1105, the lock 610 of FIG. 6 may be applied to the entry 605 of FIG. 6. Block 1105 may be omitted, as indicated by dashed line 1110: For example, if Figure 3 The data structure 325 is not accessed by multiple threads. At block 1115, Figure 1 The storage device 120 issues an I / O request to access Figure 3 The data requested in the data access request 320. Note that if the data is stored in Figure 1 Memory 115, processor software cache 305 or Figure 3 In the device local memory 310 (assuming Figure 1 The processor 110 can access Figure 3 If the device local memory 310 is used, the data can be accessed in other ways besides using I / O requests. At block 1120, the Figure 3The library 335 thread has completed accessing the data, and at block 1125, the lock 610 of Figure 6 may be released. Blocks 1120 and 1125 may be omitted, as indicated by dashed line 1130: for example, if the data structure is not accessed by multiple threads.

[0113] Fig.12 The present invention is shown in the embodiment of the present invention. Figure 1 105 machines are processed Figure 3 A flowchart of an example process for processing request 365. Fig.12 In block 1205, the Figure 3 Application 330 received Figure 3 Processing request 365. Figure 3 The processing request 325 may specify data to be processed using a specific function or procedure. At block 1210, Figure 3 The analysis engine 360 ​​may perform analysis to determine Figure 3 Should processor 110 or processor 355 execute Figure 3 Finally, at block 1215, Figure 7 The allocation unit 715 can Figure 3 Request 365 is dispatched to Figure 1 processor 110, processor 355, or both to execute in collaboration.

[0114] Fig.13 The present invention is shown in the embodiment of the present invention. Figure 1 Machine 105 receives Figure 3 A flowchart of an example process for processing request 365. Fig.13 In block 1305, Figure 3 Library 335 can intercept Figure 3 The application 330 processes the request 365 .

[0115] Figures 14A-14B The present invention is shown in the embodiment of the present invention. Figure 1 The machine 105 determines where to process Figure 3 A flowchart of an example process for processing request 365. Fig.14A In block 1405, Figure 3 The analysis engine 360 ​​may determine Figure 3 Whether the data processed by the processing request 365 is stored on the host (i.e., in Figure 3 memory 115 or processor software cache 305). Figure 3 The 360 ​​analysis engine can also determine Figure 3 Can the processor 355 execute Figure 3If the data is stored in memory 115 or processor software cache 305, or if Figure 3 The processor 355 cannot execute Figure 3 , then at block 1410, Figure 7 The allocation unit 715 can Figure 3 Requests for processing 365 are assigned to Figure 1 The processor 110 of FIG.

[0116] If the data is not stored (or not fully stored) in Figure 3 's memory 115 or processor software cache 305, then at block 1415, Figure 3 The analysis engine 360 ​​can determine whether the data is stored in Figure 1 on the storage device 120 (i.e., on Figure 3 ). If so, then at block 1420, Figure 7 The allocation unit 715 can Figure 3 Requests for processing 365 are assigned to Figure 3 Processor 355.

[0117] If the data is not completely stored on the host or Figure 1 On the storage device 120, then at block 1425, Figure 7 The calculation of the calculator 710 will be done by Figure 3 The processing request 365 is processed by moving the data from the host to the storage device 120, and at block 1430, Figure 7 The calculation of the calculator 710 will be done by Figure 3 Processing requests for 365 processed data from Figure 1 Then, at block 1435, Figure 7 The calculator 710 calculates the estimated host processing time 805, and at block 1440, Figure 7 Calculator 710 calculations Figure 8 The estimated device processing time 810. Finally, at block 1445, Figure 7 The allocation unit 715 is based on Figure 8 The estimated host processing time 805 and the estimated device processing time 810 will be Figure 3 Request 365 is dispatched to Figure 1 processor 110, processor 355, or both to collaboratively execute processing requests.

[0118] exist Figure 9-14BIn the flowchart, some embodiments of the present disclosure are shown. However, those skilled in the art will recognize that other embodiments of the present disclosure are possible by changing the order of the boxes, by omitting boxes, or by including links not shown in the drawings. All these variations of the flowchart are considered embodiments of the present disclosure, whether explicitly described or not.

[0119] Embodiments of the present disclosure may include a data structure that can identify which memory in a tiered storage system stores specific data. The data structure may be accessed when a data access request is issued to determine where the data is currently stored. The use of the data structure may provide a technical advantage in retrieving data faster.

[0120] Some embodiments of the present disclosure may also include an analysis engine. The analysis engine may determine whether a data processing request can be more efficiently executed on a host processor or in a processor associated with a storage device (such as a computational storage unit). The processing request may then be dispatched to where the processing request can be most efficiently executed, which may include collaborative execution of the processing request by a host processor and a processor in a storage device. The use of an analysis engine may provide a technical advantage in performing data processing more efficiently.

[0121] When data misses the host cache, using the host cache may result in high input / output (I / O) access time and data movement cost. Embodiments of the present disclosure can solve such problems by using scalable indexing for concurrent access to the host cache and device dynamic random access memory (DRAM).

[0122] For storage devices including coprocessors / accelerators / computing devices, concurrent data processing at both the host and the device is possible. Dynamic model-driven offload support across hosts and devices can be achieved by actively monitoring hardware and software metrics for efficient processing.

[0123] Embodiments of the present disclosure may support collaborative caching utilizing near-storage memory accessible via a cache coherent interconnect protocol such as the Compute Express Link (CXL) protocol or via the Non-Volatile Memory Express (NVMe) specification. A host-managed scalable index may map a range of blocks in a file to different caches, allowing concurrent access to these blocks. Thus, embodiments of the present disclosure may provide improved application performance and reduced central processing unit (CPU) stalls.

[0124] Embodiments of the present disclosure may include a cache manager that may handle both I / O and data processing flows simultaneously.

[0125] User-level runtime:

[0126] Applications can issue I / O requests, which can be intercepted by the runtime library. The runtime library can use a scalable interval tree to locate data in host or device RAM or storage. Cache misses can be dispatched to devices as I / O requests using an I / O queue.

[0127] Near Storage Cache Manager:

[0128] The near storage cache manager can extract the request from the I / O queue and can read the block from the storage device to the near storage cache to apply the change to the cache.For data processing operations (e.g., K nearest neighbor search), the application can use the runtime library to call predefined read-CRC-write functions.

[0129] The scalable interval tree can use a dynamic model component to decide whether to process a request in the host, in the device, or collaboratively on both. The dynamic model component can be based on an analytical method that can calculate the approximate time to process a request on the host or device before processing.

[0130] The example function (read-cal_distance_nearestK) may be executed simultaneously on both the host and the device.

[0131] Th=RdD / Bhm_dm+RsD / Bds_hm+Ehavg

[0132] Td=RhD / Bhm_dm+RsD / Bds_dm+Cmdavg*Qlen+Edavg

[0133] The above equations can be used to calculate the processing time of a request on the host (Th) and the device (Td) respectively.

[0134] dataRatio(R) may indicate that data associated with a request may be distributed across HostCache, DevCache, and storage. Ratios Rhm, Rdm, and Rs may indicate the portion of data in host memory (hm), device memory (dm), and storage (s) for each request.

[0135] Execution time (E) can capture the processing cost alone, Ehavg can represent the average time to execute a request on the host, and Edavg can represent the average time on the device.

[0136] The data transfer cost (B) may capture the data movement between HostCache, DevCache, and storage. Bhm_dm may represent the data transfer bandwidth between HostCache and DevCache, Bds_hm may represent the bandwidth between storage and HostCache, and Bds_dm may represent the bandwidth between storage and DevCache.

[0137] The queue waiting time (Qlen) can represent the completion time of the request and can depend on the time the request spends in the queue. This time can vary based on the number of regular and data processing requests in each file I / O queue and the average time required to process a request, indicated by Cmdavg*Qlen.

[0138] The following discussion is intended to provide a brief general description of suitable one or more machines in which certain aspects of the present disclosure may be implemented. One or more machines may be controlled at least in part by input from conventional input devices (such as keyboards, mice, etc.) and by instructions received from another machine, interaction with a virtual reality (VR) environment, biometric feedback, or other input signals. As used herein, the term "machine" is intended to broadly cover a single machine, a virtual machine, or a system of communicatively coupled machines, virtual machines, or devices operating together. Exemplary machines include computing devices such as personal computers, workstations, servers, portable computers, handheld devices, phones, tablets, etc., and transportation equipment such as private or public transportation, such as cars, trains, carriages, etc.

[0139] One or more machines may include embedded controllers, such as programmable or non-programmable logic devices or arrays, application specific integrated circuits, embedded computers, smart cards, etc. One or more machines may utilize one or more connections to one or more remote machines, such as through a network interface, modem, or other communication coupling. The machines may be interconnected through physical and / or logical networks, such as an intranet, the Internet, a local area network, a wide area network, etc. Those skilled in the art will appreciate that network communications may utilize a variety of wired and / or wireless short-range or long-range carriers and protocols, including radio frequency (RF), satellite, microwave, Institute of Electrical and Electronics Engineers (IEEE) 802.11, optical, infrared, cable, laser, etc.

[0140] Embodiments of the present disclosure may be described with reference to or in conjunction with associated data including functions, procedures, data structures, applications, etc., which, when accessed by a machine, cause the machine to perform a task or define an abstract data type or low-level hardware context. The associated data may be stored, for example, in volatile and / or non-volatile memory (e.g., RAM, ROM, etc.), or in other storage devices and their associated storage media, including hard drives, floppy disks, optical storage, magnetic tapes, flash memory, memory sticks, digital video disks, bio-memory, etc. The associated data may be delivered in the form of packets, serial data, parallel data, propagated signals, etc., through a transmission environment (including physical and / or logical networks), and may be used in a compressed or encrypted format. The associated data may be used in a distributed environment and stored locally and / or remotely for machine access.

[0141] Embodiments of the present disclosure may include a tangible, non-transitory machine-readable medium including instructions executable by one or more processors, the instructions including instructions for performing elements of the present disclosure as described herein.

[0142] The various operations of the above methods may be performed by any suitable unit capable of performing the operations, such as various hardware and / or software components, circuits and / or modules. Software may include an ordered list of executable instructions for implementing logical functions, and may be embodied in any "processor-readable medium" for use by or in conjunction with an instruction execution system, device or apparatus (such as a single-core or multi-core processor or a system containing a processor).

[0143] The blocks or steps of the methods or algorithms and functions described in conjunction with the embodiments disclosed herein may be directly embodied in hardware, in a software module executed by a processor, or in a combination of the two. If implemented in software, the functions may be stored as one or more instructions or codes on a tangible, non-transitory computer-readable medium or transmitted via a tangible, non-transitory computer-readable medium. The software module may reside in a random access memory (RAM), flash memory, read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disks, removable disks, CDROMs, or any other form of storage medium known in the art.

[0144] The principles of the present disclosure have been described and illustrated with reference to the illustrated embodiments, and it will be appreciated that the illustrated embodiments may be modified in arrangement and detail without departing from these principles, and the illustrated embodiments may be combined in any desired manner. Furthermore, although the foregoing discussion has focused on specific embodiments, other configurations are contemplated. In particular, even though expressions such as "according to embodiments of the present disclosure" are used herein, these phrases generally refer to embodiment possibilities and are not intended to limit the present disclosure to specific embodiment configurations. As used herein, these terms may refer to the same or different embodiments that may be combined into other embodiments.

[0145] The foregoing illustrative embodiments should not be construed as limiting the disclosure thereof. Although several embodiments have been described, it will be readily appreciated by those skilled in the art that many modifications may be made to those embodiments without substantially departing from the novel teachings and advantages of the present disclosure. Therefore, all such modifications are intended to be included within the scope of the present disclosure as defined in the claims.

[0146] The embodiments of the present disclosure can be extended to the following statements without limitation:

[0147] Statement 1. An embodiment of the present disclosure includes a system, comprising:

[0148] processor;

[0149] a first memory connected to the processor;

[0150] a second memory connected to the processor; and

[0151] A data structure includes at least an entry, the entry identifying data stored in a location, the location including one of the first memory or the second memory.

[0152] Statement 2. Embodiments of the present disclosure include a system according to statement 1, wherein the first memory comprises a processor cache or a main memory.

[0153] Statement 3. An embodiment of the present disclosure includes a system according to statement 1, wherein the second memory comprises device local memory or device persistent storage.

[0154] Statement 4. An embodiment of the present disclosure comprises a system according to statement 1, further comprising a device comprising a second memory.

[0155] Statement 5. An embodiment of the present disclosure includes a system according to statement 1, further comprising a library for intercepting a data access request from an application running on a processor to access data.

[0156] Statement 6. An embodiment of the present disclosure includes a system according to statement 1, wherein:

[0157] The data structure includes a scalable interval tree; and

[0158] An entry comprises a node in the scalable interval tree.

[0159] Statement 7. An embodiment of the present disclosure includes a system according to statement 1, wherein the data structure is stored in the first memory.

[0160] Statement 8. An embodiment of the present disclosure includes a system according to statement 1, wherein the data structure is associated with the file.

[0161] Statement 9. An embodiment of the present disclosure includes the system of statement 8, further comprising a second data structure, the second data structure including at least a second entry, the second data structure being associated with the second file.

[0162] Statement 10. An embodiment of the present disclosure includes the system of statement 1, wherein the entry includes a lock.

[0163] Statement 11. An embodiment of the present disclosure includes the system of statement 10, wherein the lock is associated with a thread of an application running on the processor, the thread requesting access to the data.

[0164] Statement 12. An embodiment of the present disclosure includes a system according to statement 1, wherein:

[0165] The processor includes a first thread running on the processor and a second thread running on the processor;

[0166] The data structure includes at least a second entry, the second entry identifying a second location of the second data; and

[0167] The data structure is configured to support access by a first thread to the data from the location and access by a second thread to a second data from a second location.

[0168] Statement 13. An embodiment of the present disclosure includes a system according to statement 12, wherein the data structure is configured to support a first thread accessing the data from the location and a second thread accessing the second data from the second location in parallel.

[0169] Statement 14. An embodiment of the present disclosure includes the system according to statement 1, and also includes a device, wherein the device includes:

[0170] a second memory; and

[0171] Second processor.

[0172] Statement 15. An embodiment of the present disclosure includes a system according to statement 14, further comprising an analysis engine for calculating a first estimated time on the processor regarding a processing request for target data from an application running on the processor and a second estimated time on a second processor regarding a processing request from an application running on the processor.

[0173] Statement 16. An embodiment of the present disclosure comprises a system according to statement 15, further comprising a library for intercepting a processing request from an application running on a processor.

[0174] Statement 17. An embodiment of the present disclosure includes the system of statement 15, wherein the analysis engine is configured to assign processing requests from applications running on the processor to the processor based at least in part on the target data in the first memory.

[0175] Statement 18. An embodiment of the present disclosure includes a system according to statement 15, wherein the analysis engine is configured to allocate processing requests from an application running on a processor to the processor based at least in part on a second processor not executing the processing request from the application running on the processor.

[0176] Statement 19. An embodiment of the present disclosure includes the system of statement 15, wherein the analysis engine is configured to assign a processing request from an application running on the processor to the second processor based at least in part on the target data being in the second memory.

[0177] Statement 20. An embodiment of the present disclosure includes the system of statement 15, wherein the analysis engine is configured to:

[0178] determining that a first portion of the target data is stored in a first memory and a second portion of the target data is stored in a second memory;

[0179] calculating a first estimated time for the processor to execute a processing request from an application running on the processor; and

[0180] A second estimated time for a second processor to execute a processing request from an application running on the processor is calculated.

[0181] Statement 21. An embodiment of the present disclosure includes a system according to statement 20, wherein the analysis engine is further configured to dispatch a processing request from an application running on a processor to the processor, a second processor, or both the processor and the second processor.

[0182] Statement 22. An embodiment of the present disclosure includes the system of statement 20, wherein the analysis engine is further configured to:

[0183] calculating a first transfer time to transfer the second portion of the target data to the first memory; and

[0184] A second transfer time is calculated to transfer the first portion of the data to the second memory.

[0185] Statement 23. An embodiment of the present disclosure includes a method comprising:

[0186] receiving a data access request for data from an application running on the processor;

[0187] identifying a data structure based on a data access request;

[0188] identifying an entry in a data structure based on a data access request;

[0189] identifying a location to store the data based on the entry in the data structure; and

[0190] access data from said location,

[0191] The location includes the first memory or the second memory.

[0192] Statement 24. An embodiment of the present disclosure includes a method according to statement 23, wherein receiving a data access request for data from an application running on the processor includes intercepting a data access request for data from an application running on the processor.

[0193] Statement 25. An embodiment of the present disclosure includes a method according to statement 24, wherein intercepting a data access request to the data from an application running on the processor includes intercepting, by a library, a data access request to the data from an application running on the processor.

[0194] Statement 26. An embodiment of the present disclosure includes the method of statement 23, wherein identifying the data structure based on the data access request includes identifying a scalable interval tree based on the data access request.

[0195] Statement 27. An embodiment of the present disclosure includes the method of statement 26, wherein identifying an entry in a data structure based on a data access request includes identifying a node in a scalable interval tree based on the data access request.

[0196] Statement 28. An embodiment of the present disclosure includes a method according to statement 23, wherein:

[0197] Receiving a data access request for data from an application running on the processor includes receiving a second data access request for second data from the application running on the processor;

[0198] Identifying the data structure based on the data access request includes identifying the data structure based on the second data access request;

[0199] Identifying an entry in the data structure based on the data access request includes identifying a second entry in the data structure based on the second data access request; and

[0200] Identifying a location to store the data based on the entry in the data structure includes identifying a location to store second data based on a second entry in the data structure.

[0201] Statement 29. An embodiment of the present disclosure includes a method according to statement 28, wherein identifying a location to store data based on an entry in a data structure includes: identifying a location to store second data based on a second entry in the data structure in parallel with identifying a location to store data based on an entry in the data structure.

[0202] Statement 30. An embodiment of the present disclosure includes the method of statement 23, wherein the first memory comprises a processor cache, a main memory, a device local memory, or a device persistent storage.

[0203] Statement 31. An embodiment of the present disclosure includes the method of statement 23, wherein the second memory comprises a processor cache, a main memory, a device local memory, or a device persistent storage.

[0204] Statement 32. An embodiment of the present disclosure includes the method of statement 23, wherein accessing the data from the location comprises accessing, by the processor, the data from the location.

[0205] Statement 33. An embodiment of the present disclosure includes the method of statement 23, wherein accessing the data from the location comprises issuing an input / output (I / O) request to a device, the device comprising the first memory.

[0206] Statement 34. An embodiment of the present disclosure includes a method according to statement 23, wherein:

[0207] Receiving a data access request for data from an application running on the processor includes receiving a data access request for data from a thread of an application running on the processor; and

[0208] Accessing data from the location includes applying a lock to an entry in the data structure for use by the thread.

[0209] Statement 35. An embodiment of the present disclosure includes the method of statement 34, further comprising releasing a lock on an entry in a data structure for use by the thread.

[0210] Statement 36. An embodiment of the present disclosure includes a method according to statement 35, wherein:

[0211] The method further includes: receiving a notification that the thread has used data from the first memory; and

[0212] Releasing the lock on the entry in the data structure for use by the thread includes releasing the lock on the entry in the data structure for use by the thread based at least in part on receiving notification that the thread has used data from the first memory.

[0213] Statement 37. An embodiment of the present disclosure comprises the method according to statement 23, further comprising returning the data to the processor.

[0214] Statement 38. An embodiment of the present disclosure includes a method according to statement 23, wherein:

[0215] The data access request requests data from the file; and

[0216] Data structures are used for files.

[0217] Statement 39. An embodiment of the present disclosure includes a method according to statement 38, wherein the second data structure is for the second file.

[0218] Statement 40. An embodiment of the present disclosure includes a method comprising:

[0219] receiving a processing request from an application running on the first processor, the processing request to be applied to data;

[0220] performing analysis of the processing request to determine a target for executing the processing request, the target comprising a first processor or a second processor associated with the device; and

[0221] Dispatches processing requests to targets.

[0222] Statement 41. An embodiment of the present disclosure includes a method according to statement 40, wherein receiving a processing request from an application running on the first processor includes intercepting a processing request for data from an application running on the first processor.

[0223] Statement 42. An embodiment of the present disclosure includes a method according to statement 41, wherein intercepting a request for processing data from an application running on the first processor includes intercepting, by a library, a request for processing data from an application running on the first processor.

[0224] Statement 43. An embodiment of the present disclosure includes a method according to statement 40, wherein:

[0225] Performing analysis of the processing request to determine a target for executing the processing request includes determining that data is stored in a memory associated with the first processor; and

[0226] Dispatching the processing request to the target includes dispatching the processing request to the first processor.

[0227] Statement 44. An embodiment of the present disclosure includes a method according to statement 40, wherein:

[0228] performing an analysis of the processing request to determine a target for executing the processing request includes determining that the second processor is not configured to execute the processing request; and

[0229] Dispatching the processing request to the target includes dispatching the processing request to the first processor.

[0230] Statement 45. An embodiment of the present disclosure includes a method according to statement 40, wherein:

[0231] performing analysis of the processing request to determine a target for executing the processing request includes determining that data is stored in a memory associated with the second processor; and

[0232] Dispatching the processing request to the target includes dispatching the processing request to the second processor.

[0233] Statement 46. An embodiment of the present disclosure includes a method according to statement 40, wherein performing an analysis of the processing request to determine a target for executing the processing request includes:

[0234] determining that a first portion of the data is stored in a first memory associated with the first processor and a second portion of the data is stored in a second memory associated with the second processor;

[0235] calculating a first estimated time for the first processor to execute the processing request; and

[0236] A second estimated time for the second processor to execute the processing request is calculated.

[0237] Statement 47. An embodiment of the present disclosure includes a method according to statement 46, wherein dispatching the processing request to the target includes dispatching the processing request to the first processor, the second processor, or both the first processor and the second processor based at least in part on the first estimated time and the second estimated time.

[0238] Statement 48. An embodiment of the present disclosure includes a method according to statement 46, wherein:

[0239] Calculating a first estimated time for the first processor to perform the processing request includes calculating a first transfer time to transfer the second portion of the data to a first memory associated with the first processor; and

[0240] Calculating a second estimated time for the second processor to perform the processing request includes calculating a second transfer time to transfer the first portion of the data to a second memory associated with the second processor.

[0241] Statement 49. Embodiments of the present disclosure include an article of manufacture comprising a non-transitory storage medium having stored thereon instructions that, when executed by a machine, result in:

[0242] receiving a data access request for data from an application running on the processor;

[0243] identifying a data structure based on a data access request;

[0244] identifying an entry in a data structure based on a data access request;

[0245] identifying a location to store the data based on the entry in the data structure; and

[0246] accessing said data from said location,

[0247] The location includes the first memory or the second memory.

[0248] Statement 50. An embodiment of the present disclosure includes an article of manufacture according to statement 49, wherein receiving a data access request for data from an application running on the processor includes intercepting a data access request for data from an application running on the processor.

[0249] Statement 51. An embodiment of the present disclosure includes an article of manufacture according to statement 50, wherein intercepting a data access request to the data from an application running on the processor comprises intercepting, by a library, a data access request to the data from an application running on the processor.

[0250] Statement 52. An embodiment of the present disclosure includes an article of manufacture according to statement 49, wherein identifying the data structure based on the data access request includes identifying a scalable interval tree based on the data access request.

[0251] Statement 53. An embodiment of the present disclosure includes an article of manufacture according to statement 52, wherein identifying an entry in a data structure based on a data access request comprises identifying a node in a scalable interval tree based on the data access request.

[0252] Statement 54. Embodiments of the present disclosure include an article according to statement 49, wherein:

[0253] Receiving a data access request for data from an application running on the processor includes receiving a second data access request for second data from an application running on the processor;

[0254] Identifying the data structure based on the data access request includes identifying the data structure based on the second data access request;

[0255] Identifying an entry in the data structure based on the data access request includes identifying a second entry in the data structure based on the second data access request; and

[0256] Identifying a location to store the data based on the entry in the data structure includes identifying a location to store second data based on a second entry in the data structure.

[0257] Statement 55. An embodiment of the present disclosure includes an article according to statement 54, wherein identifying a location to store data based on an entry in a data structure includes identifying a location to store second data based on a second entry in the data structure in parallel with identifying a location to store data based on the entry in the data structure.

[0258] Statement 56. Embodiments of the present disclosure include an article of manufacture according to statement 49, wherein the first memory comprises a processor cache, a main memory, a device local memory, or a device persistent storage.

[0259] Statement 57. Embodiments of the present disclosure include an article of manufacture according to statement 49, wherein the second memory comprises a processor cache, main memory, device local memory, or device persistent storage.

[0260] Statement 58. An embodiment of the present disclosure includes an article of manufacture according to statement 49, wherein accessing the data from the location comprises accessing the data from the location by a processor.

[0261] Statement 59. An embodiment of the present disclosure includes an article of manufacture according to statement 49, wherein accessing the data from the location comprises issuing an input / output (I / O) request to a device, the device comprising the first memory.

[0262] Statement 60. Embodiments of the present disclosure include an article according to statement 49, wherein:

[0263] Receiving a data access request for data from an application running on the processor includes receiving a data access request for data from a thread of the application running on the processor; and

[0264] Accessing data from the location includes applying a lock to an entry in the data structure for use by the thread.

[0265] Statement 61. An embodiment of the present disclosure includes an article of manufacture according to statement 60, the non-transitory storage medium having further instructions stored thereon that, when executed by a machine, result in releasing a lock on an entry in a data structure for use by a thread.

[0266] Statement 62. Embodiments of the present disclosure include an article according to statement 61, wherein:

[0267] The non-transitory storage medium has further instructions stored thereon that, when executed by the machine, result in: receiving a notification that a thread has used data from a first memory; and

[0268] Releasing the lock on the entry in the data structure for use by the thread includes releasing the lock on the entry in the data structure for use by the thread based at least in part on receiving notification that the thread has used data from the first memory.

[0269] Statement 63. Embodiments of the present disclosure include the article of manufacture of statement 49, the non-transitory storage medium having further instructions stored thereon that, when executed by a machine, result in data being returned to the processor.

[0270] Statement 64. Embodiments of the present disclosure include an article according to statement 49, wherein:

[0271] The data access request requests data from the file; and

[0272] Data structures are used for files.

[0273] Statement 65. An embodiment of the present disclosure includes an article of manufacture according to statement 64, wherein the second data structure is for the second file.

[0274] Statement 66. Embodiments of the present disclosure include an article of manufacture comprising a non-transitory storage medium having stored thereon instructions that, when executed by a machine, result in:

[0275] receiving a processing request from an application running on the first processor, the processing request to be applied to data;

[0276] performing analysis of the processing request to determine a target for executing the processing request, the target comprising a first processor or a second processor associated with the device; and

[0277] Dispatches processing requests to targets.

[0278] Statement 67. An embodiment of the present disclosure includes an article of manufacture according to statement 66, wherein receiving a processing request from an application running on the first processor includes intercepting a processing request for data from an application running on the first processor.

[0279] Statement 68. An embodiment of the present disclosure includes an article of manufacture according to statement 67, wherein intercepting a request for processing data from an application running on the first processor includes intercepting, by a library, a request for processing data from an application running on the first processor.

[0280] Statement 69. Embodiments of the present disclosure include an article according to statement 66, wherein:

[0281] Performing analysis of the processing request to determine a target for executing the processing request includes determining that data is stored in a memory associated with the first processor; and

[0282] Dispatching the processing request to the target includes dispatching the processing request to the first processor.

[0283] Statement 70. Embodiments of the present disclosure include an article according to statement 66, wherein:

[0284] performing an analysis of the processing request to determine a target for executing the processing request includes determining that the second processor is not configured to execute the processing request; and

[0285] Dispatching the processing request to the target includes dispatching the processing request to the first processor.

[0286] Statement 71. Embodiments of the present disclosure include an article according to statement 66, wherein:

[0287] performing analysis of the processing request to determine a target for executing the processing request includes determining that data is stored in a memory associated with the second processor; and

[0288] Dispatching the processing request to the target includes dispatching the processing request to the second processor.

[0289] Statement 72. An embodiment of the present disclosure includes an article of manufacture according to statement 66, wherein performing analysis of the processing request to determine a target for executing the processing request comprises:

[0290] determining that a first portion of the data is stored in a first memory associated with the first processor and a second portion of the data is stored in a second memory associated with the second processor;

[0291] calculating a first estimated time for the first processor to execute the processing request; and

[0292] A second estimated time for the second processor to execute the processing request is calculated.

[0293] Statement 73. An embodiment of the present disclosure includes an article of manufacture according to statement 72, wherein dispatching the processing request to the target includes dispatching the processing request to the first processor, the second processor, or both the first processor and the second processor based at least in part on the first estimated time and the second estimated time.

[0294] Statement 74. Embodiments of the present disclosure include an article according to statement 72, wherein:

[0295] Calculating a first estimated time for the first processor to perform the processing request includes calculating a first transfer time to transfer the second portion of the data to a first memory associated with the first processor; and

[0296] Calculating a second estimated time for the second processor to perform the processing request includes calculating a second transfer time to transfer the first portion of the data to a second memory associated with the second processor.

[0297] Therefore, in view of the various arrangements of the embodiments described herein, this detailed description and accompanying material are intended to be illustrative only and should not be taken as limiting the scope of the disclosure. Therefore, the disclosure claims all such modifications that may fall within the scope and spirit of the appended claims and their equivalents.

Claims

1. A system for collaborative caching, include: processor; a first memory connected to the processor; a second memory connected to the processor; as well as A data structure includes at least an entry, the entry identifying data stored in a location, the location including one of the first memory or the second memory.

2. The system according to claim 1, further comprising: include: A library for intercepting data access requests from applications running on the processor to access data.

3. The system according to claim 1, in: The data structure comprises a scalable interval tree; and An entry comprises a node in the scalable interval tree.

4. The system according to claim 1, wherein The entry includes a lock; and The lock is associated with a thread of an application running on the processor, the thread requesting access to data.

5. The system according to claim 1 further includes an analysis engine, which calculates: a first estimated time for a processing request for target data on the processor from an application running on the processor; and a second estimated time for a processing request on a second processor from an application running on the processor.

6. The system according to claim 5, further comprising: include: A library that intercepts processing requests from applications running on the processor.

7. The system according to claim 5, in, The analysis engine is configured to: determining that a first portion of the target data is stored in a first memory and a second portion of the target data is stored in a second memory; calculating a first estimated time for the processor to execute a processing request from an application running on the processor; as well as A second estimated time for a second processor to execute a processing request from an application running on the processor is calculated.

8. A method for collaborative caching, include: receiving a data access request for data from an application running on the processor; identifying a data structure based on a data access request; identifying an entry in a data structure based on a data access request; identifying a location to store the data based on an entry in the data structure; as well as access data from said location, The location includes the first memory or the second memory.

9. The method according to claim 8, in, Receiving a data access request for data from an application running on the processor includes intercepting a data access request for data from an application running on the processor.

10. The method according to claim 9, in, Intercepting a data access request for data from an application running on the processor includes: intercepting, by a library, a data access request for data from an application running on the processor.

11. The method according to claim 8, in: Receiving a data access request for data from an application running on the processor includes receiving a second data access request for second data from the application running on the processor; Identifying the data structure based on the data access request includes identifying the data structure based on the second data access request; Identifying an entry in the data structure based on the data access request includes identifying a second entry in the data structure based on the second data access request; as well as Identifying a location to store the data based on the entry in the data structure includes identifying a location to store second data based on a second entry in the data structure.

12. The method according to claim 8, in, Accessing the data from the location includes accessing, by the processor, the data from the location.

13. The method according to claim 8, in, Accessing data from the location includes issuing an input / output (I / O) request to a device, the device including a first memory.

14. The method according to claim 8, in: Receiving a data access request for data from an application running on the processor includes receiving a data access request for data from a thread of the application running on the processor; as well as Accessing data from the location includes applying a lock to an entry in the data structure for use by the thread.

15. A method for collaborative caching, include: receiving a processing request from an application running on the first processor, the processing request to be applied to data; performing an analysis of the processing request to determine a target for executing the processing request, the target comprising a first processor or a second processor associated with the device; as well as Dispatches processing requests to targets.

16. The method according to claim 15, in, Receiving a processing request from an application running on the first processor includes intercepting a processing request for data from the application running on the first processor.

17. The method according to claim 15, in: performing an analysis of the processing request to determine a target for executing the processing request includes determining that data is stored in a memory associated with the first processor; and Dispatching the processing request to the target includes dispatching the processing request to the first processor.

18. The method according to claim 15, in, Analysis of the processing request is performed to determine the objectives for executing the processing request including: determining that a first portion of the data is stored in a first memory associated with the first processor and a second portion of the data is stored in a second memory associated with the second processor; calculating a first estimated time for the first processor to execute the processing request; and A second estimated time for the second processor to execute the processing request is calculated.

19. The method according to claim 18, in, Dispatching the processing request to the target includes dispatching the processing request to the first processor, the second processor, or both the first processor and the second processor based at least in part on the first estimated time and the second estimated time.

20. The method according to claim 18, in: Calculating a first estimated time for the first processor to perform the processing request includes calculating a first transfer time to transfer the second portion of the data to a first memory associated with the first processor; as well as Calculating a second estimated time for the second processor to perform the processing request includes calculating a second transfer time to transfer the first portion of the data to a second memory associated with the second processor.