System and method for prefetching data
By introducing a prefetch engine in the processor, prefetching data from nonvolatile storage media into volatile storage media, the storage device access latency problem is solved and system performance and responsiveness is improved.
Patent Information
- Application Number
- CN202410553005.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-07-28
- Filing Date
- 2024-05-07
- Publication Date
- 2024-11-12
AI Technical Summary
There is a delay in accessing storage devices, especially non-volatile storage media has higher latency than volatile storage media, which affects system performance and responsiveness.
By introducing a prefetch engine in the processor, prefetching data from nonvolatile storage media into volatile storage media using prefetch instructions and storing it in the processor cache, using speculative and conventional memory reads Strategy to reduce latency.
Effectively reduces memory access latency, improves overall system performance and responsiveness, and does not require special operating system or data structure support.
Smart Images

Figure CN118939180A_ABST
Abstract
Description
Technical Field
[0001] One or more aspects according to embodiments of the present disclosure relate to a storage device, and more particularly, to pre-fetching data stored in the storage device. Background Art
[0002] Applications can interact with storage or memory devices (collectively referred to as storage devices) to read (or load) and write (or store) data. There is often latency when accessing storage devices. The type of latency involved can depend on the storage media included in the storage device. Some storage media have lower latency than other storage media. Therefore, it may be desirable to predict data that will be accessed in the near future and store the data in a lower latency storage medium to improve overall system performance and responsiveness.
[0003] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art. Summary of the invention
[0004] One or more embodiments of the present disclosure relate to a computing system, which includes a storage device, a processor configured to communicate with the storage device, and a memory coupled to the processor. The storage device includes a first storage medium and a second storage medium. The memory stores instructions that, when executed by the processor, cause the processor to: identify a first address; identify a first setting associated with the first address; and issue a first command to the first storage medium based on the first setting. The first command is used to perform a first type of memory read. The storage device is configured to retrieve first data associated with a first address in the first storage medium to the second storage medium based on the first command.
[0005] In some embodiments, the first address is identified in a prefetch instruction included in a program executed by the processor.
[0006] In some embodiments, the first storage medium includes non-volatile memory and the second storage medium includes volatile memory.
[0007] In some embodiments, the first type of memory read includes issuing a first command to read data in a first address, wherein the processor is configured to issue a second command for a second function during an interval in which the first command is pending.
[0008] In some embodiments, the instructions further cause the processor to: identify a second address; identify a second setting associated with the second address; and issue a second command to the memory based on the second setting, wherein the second command is used to perform a second type of memory read different from the first type of memory read, wherein the memory is configured to retrieve second data associated with the second address based on the second command.
[0009] In some embodiments, the second type of memory read comprises a blocking memory read of the second address.
[0010] In some embodiments, the storage device includes a queue for storing the first address.
[0011] In some embodiments, a storage device includes a controller configured to: retrieve a first address from a queue; retrieve first data from a first storage medium based on the first address retrieved from the queue; and store the first data to a second storage medium.
[0012] In some embodiments, the first setting identifies a first pre-fetch strategy, wherein the first pre-fetch strategy is based on an access latency associated with the first storage device.
[0013] In some embodiments, the first setting is stored in a table containing a mapping of virtual addresses to physical addresses.
[0014] One or more embodiments of the present disclosure relate to a method, comprising: identifying a first address by a processor that communicates with a storage device; identifying a first setting associated with the first address by the processor; and issuing a first command by the processor to a first storage medium of the storage device based on the first setting, wherein the first command is used to perform a first type of memory read, and wherein the storage device is configured to retrieve first data associated with the first address in the first storage medium to a second storage medium of the storage device based on the first command.
[0015] These and other features, aspects and advantages of the embodiments of the present disclosure will be more fully understood when considered with reference to the following detailed description, appended claims and accompanying drawings.Of course, the actual scope of the invention is defined by the appended claims. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Non-limiting and non-exhaustive embodiments of the present invention are described with reference to the following figures, wherein like reference numerals refer to like parts throughout the various views unless otherwise specified.
[0017] Figure 1 depicts a block diagram of a system for prefetching data into a central processing unit (CPU) cache in accordance with one or more embodiments;
[0018] Figure 2 depicts a block diagram of a storage controller according to one or more embodiments;
[0019] Figure 3 depicts a conceptual diagram of a CPU prefetch engine that employs different prefetch strategies based on information stored in a translation lookaside buffer in accordance with one or more embodiments;
[0020] Figure 4 depicts an example data flow for pre-fetching data from a storage device according to one or more embodiments;
[0021] Figure 5 A flowchart depicting a process for pre-fetching data from a storage device into a CPU cache according to one or more embodiments; and
[0022] Figure 6 Depicted is a flow diagram of a process for processing a memory read command from a host in accordance with one or more embodiments. DETAILED DESCRIPTION
[0023] Hereinafter, example embodiments will be described in more detail with reference to the accompanying drawings, in which the same reference numerals always refer to the same elements. However, the present disclosure may be embodied in various different forms and should not be construed as being limited to the embodiments shown herein. On the contrary, these embodiments are provided as examples so that the present disclosure will be thorough and complete, and will fully convey the aspects and features of the present disclosure to those skilled in the art. Therefore, processes, elements and techniques that are not necessary for a complete understanding of the aspects and features of the present disclosure for those of ordinary skill in the art may not be described. Unless otherwise stated, throughout the drawings and written descriptions, the same reference numerals represent the same elements, and therefore, their descriptions are not repeated. In addition, in the drawings, for clarity, the relative sizes of elements, layers and regions may be enlarged and / or simplified.
[0024] Embodiments of the present disclosure are described below with reference to block diagrams and flow charts. Therefore, it should be understood that each block of block diagrams and flow charts can be implemented in the form of computer program products, complete hardware embodiments, combinations of hardware and computer program products and / or devices, systems, computing devices, computing entities and / or executing interchangeable instructions, operations, steps and similar words (such as executable instructions, execution instructions, program codes, etc.) on computer-readable storage media. For example, the retrieval, loading and execution of code can be performed sequentially so that one instruction is retrieved, loaded and executed at a time. In some example embodiments, retrieval, loading and / or execution can be performed in parallel so that multiple instructions are retrieved, loaded and / or executed together. Therefore, these embodiments can produce a specially configured machine that performs the steps or operations specified in the block diagrams and flow charts. Therefore, block diagrams and flow charts support various combinations of embodiments for executing specified instructions, operations or steps.
[0025] Generally speaking, an application running on a host computing device (referred to as a "host") may need to store data to and load data from a storage device. There is typically a delay when accessing a storage device. Depending on the storage medium storing the data to be retrieved, the delays involved may be different. For example, a storage device may have both a volatile storage medium (e.g., dynamic random access memory (DRAM)) and a non-volatile storage medium (e.g., NAND flash memory). The latency of a volatile storage medium may be lower than the latency of a non-volatile storage medium. It may be desirable to use a volatile storage medium to store blocks, groups of blocks, or pages (collectively referred to as "blocks") of data that are expected to be accessed by the host in the near future.
[0026] In general, embodiments of the present disclosure are directed to systems and methods for prefetching data from a storage device into a central processing unit (CPU) cache memory of a host. In some embodiments, the host includes a host prefetch engine configured to receive software prefetch instructions. The software prefetch instructions can be generated by a compiler during the compilation of the program and inserted into the program. In some embodiments, the host may also include a hardware prefetcher that can identify a memory location to be prefetched via a hardware mechanism that may not require explicit prefetch instructions to be inserted into the program.
[0027] In some embodiments, the host prefetch engine is configured to prefetch data from an attached storage device (e.g., a persistent storage device) in a manner that minimizes data access latency associated with the storage device. In some embodiments, data can also be prefetched from the host main memory (e.g., host DRAM) into the CPU cache. The same prefetch instruction can be used to prefetch data from an attached storage device or the host main memory. Prefetching data into the CPU cache before the data is used by the program (e.g., for calculation) can help reduce memory access latency and improve overall system responsiveness.
[0028] In some embodiments, the processing of prefetch instructions (e.g., instructions inserted into a program) may vary depending on latency and / or the type of storage device from which data is prefetched. For example, prefetching data from an attached storage device via a memory read request may block the host processor from performing other tasks while waiting for the data to return. Due to the latency that may be involved in retrieving data from an attached storage device, prefetching data from the device via a conventional memory read request may result in inefficient CPU usage. This concern may be reduced when prefetching data from a host DRAM because the host DRAM may have lower access latency than an attached storage device.
[0029] In some embodiments, the host prefetch engine is configured to perform non-blocking memory reads (hereinafter referred to as speculative memory reads), which can allow the host processor to issue a read (or load) command without waiting for data to be returned in response to the command. In some embodiments, the speculative memory read is processed by the attached storage device as a prefetch hint to prefetch data from the device's non-volatile memory (NVM) into the device's cache memory. The host can perform a regular (e.g., blocking) memory read after the speculative memory read to read the prefetched data from the device's cache memory into the CPU cache. Because the prefetched data can reside in the device's cache memory when the regular memory read is issued, the access latency of the regular memory read can be reduced.
[0030] In some embodiments, when prefetching data from main memory, the host prefetch engine uses conventional memory reads. In this regard, it may not be necessary to issue speculative memory reads to main memory, because retrieving data from main memory is usually very fast.
[0031] In some embodiments, the host prefetch engine determines the prefetch strategy to use based on the address of the memory block to be prefetched. For example, if the memory block resides in an attached storage device with an access latency above a given threshold, a speculative prefetch strategy may be employed (e.g., a speculative memory read followed by a regular memory read). If the memory block resides in main memory with an access latency below a given threshold, a regular prefetch strategy may be employed (e.g., a regular memory read without a previous speculative memory read).
[0032] As discussed, prefetching data from NVM to device cache via speculative memory reads helps reduce memory access latency during regular memory reads, which helps improve overall system performance and responsiveness. Users can also leverage existing compilers to insert prefetch instructions as usual and leave it to the CPU prefetch engine to determine the appropriate prefetch strategy to use. This can eliminate the need to expose special data structures to the user or eliminate the need for special operating system (OS) or runtime support in order to implement speculative prefetch strategies.
[0033] Figure 1 A block diagram of a system for prefetching data into a CPU cache in accordance with one or more embodiments is depicted. The system may include a host computing device ("host") 100 coupled to an attached storage device 102 via one or more data communication links 104. In some embodiments, data communication links 104 may include various common interfaces such as, for example, Ethernet, a universal serial bus (USB), and / or any wired or wireless data communication link.
[0034] The host 100 may include a processor 106, a main memory 108, and a host interface 110. The processor 106 may be configured to run one or more applications 112 based on instructions stored in the main memory 108. The application 112 may be any application configured to send a request (e.g., a data access request) to the storage device 102. For example, the application 112 may be a big data analysis application, an e-commerce application, a database application, a machine learning application, etc. Data requests, such as load and store requests, may be generated during the execution of the application 112. The application 112 may use the results of the data requests to generate outputs.
[0035] The application 112 may communicate with the storage device 102 via the host interface 110. The host interface 110 may include a physical connection as well as software instructions that may be executed by the processor 106. In some embodiments, the host interface 110 allows the host 100 and the storage device 102 to send and receive data using a protocol such as, for example, CXL, although the embodiments are not limited thereto. The performance of the application 112 may depend on the speed at which the requested data is retrieved by the storage device 102 and returned to the application.
[0036] In addition to or instead of CXL, the host interface 110 may use other protocols such as Cache Coherent Interconnect for Accelerators (CCIX), a dual in-line memory module (DIMM) interface, a small computer system interface (SCSI), a non-volatile memory interconnect express (NVMe), a peripheral component interconnect express (PCIe), Ethernet remote direct memory access (RDMA), a serial advanced technology attachment (SATA), a Fibre Channel, a serial attached SCSI (SAS), NVMe over fiber (NVMe-oF), an iWARP protocol, an InfiniBand protocol, a 5G wireless protocol, a Wi-Fi protocol, a Bluetooth protocol, and the like.
[0037] In some embodiments, the processor 106 is further configured with a CPU prefetch engine 114, which is configured to prefetch data from the storage device 102 or the main memory 108 into the CPU cache 124. In some embodiments, the CPU cache 124 may include a first level cache (L1), a second level cache (L2), and / or a last level cache (LLC).
[0038] The prefetch strategy used by the CPU prefetch engine 114 to prefetch data into the CPU cache 124 may depend on the access latency associated with the storage device 102 if the data is to be prefetched from the storage device 102, or the access latency associated with the main memory 108 if the data is to be prefetched from the memory. In some embodiments, if the data to be prefetched is located in the storage device 102 associated with the first access latency, a speculative prefetch strategy is employed. The speculative prefetch strategy may include sending a speculative (non-blocking) memory read command followed by a regular (blocking) memory read. The regular memory read may occur after the data prefetch in response to the speculative memory read has been completed.
[0039] In some embodiments, speculative memory reads are non-blocking because they do not require the storage device 102 to return data immediately in response to the speculative read command, and the CPU does not wait for the device to return data. With speculative memory reads, the CPU does not need data to continue execution. More specifically, speculative memory reads may not stall the CPU pipeline, and other commands in the pipeline for other functions may continue to execute during the interval while the speculative read command is waiting to complete (e.g., waiting for data to be returned).
[0040] In some embodiments, a regular memory read is blocked because the CPU is waiting for the storage device 102 to return data to execute the instruction. In the event that the storage device does not return data, the CPU pipeline may be stalled, thereby preventing the execution of other functions while waiting for the read command to complete (e.g., waiting for data to be returned in response to the read command). The time that the CPU pipeline is stalled may depend on the speed at which the data is returned.
[0041] Speculative memory reads and regular memory reads may be sent using, for example, the CXL.mem protocol.In some embodiments, if the data to be prefetched is located in main memory 108 having a second access latency lower than the first access latency, a regular prefetch strategy is employed.
[0042] In some embodiments, a translation lookaside buffer (TLB) and / or a page table (collectively referred to as TLB 126) may be extended to store a prefetch policy to be used in an entry of the TLB. For example, a prefetch policy bit in the entry may be set to "1" if a speculative prefetch policy is to be used to prefetch data stored at an address represented by an entry, or to "0" if a conventional prefetch policy is to be used. In some embodiments, in the event of a TLB miss, the page table is checked for a page table entry, and the TLB entry and the corresponding prefetch policy bit are populated from the page table entry.
[0043] Storage device 102 may take the form of a solid state drive (SSD), persistent memory, and / or other attached storage devices. In some embodiments, storage device 102 includes (or is embodied as) an SSD with cache coherence and / or compute capabilities. In some embodiments, storage device 102 includes (or is embodied as) a dual-mode persistent memory with dual-mode access to its storage space. In a first mode, storage device 102 is accessible as disk storage (e.g., via CXL.io). In a second mode, storage device 102 is accessible as device-attached storage (e.g., via CXL.mem or CXL.cache).
[0044] In some embodiments, the storage device 102 includes a storage controller 116, a storage memory 118, and a non-volatile memory (NVM) 120. The storage memory 118 may be a high-performance memory of the storage device 102, and may include (or may be) a volatile memory, such as DRAM, for example, but the present disclosure is not limited thereto, and the storage memory 118 may be any suitable type of high-performance volatile or non-volatile memory. Although a single storage memory 118 is described for simplicity, those skilled in the art will recognize that the storage device 102 may include other local memories for temporarily storing other data of the storage device.
[0045] In some embodiments, storage memory 118 is used and managed as a cache memory. In this regard, storage memory (also referred to as device cache memory) 118 can store a copy of data stored in NVM 120. For example, data that application 112 is to access in the near future can be copied from NVM 120 to storage memory 118 to allow data to be retrieved from storage memory 118 instead of NVM 120. In some embodiments, storage memory 118 has a lower access latency than NVM 120. Therefore, in some embodiments, accessing data from storage memory 118 helps improve overall system performance and responsiveness.
[0046] NVM 120 may persistently store data received, for example, from host 100. NVM 120 may include, for example, one or more NAND flash memories, but the present disclosure is not limited thereto, and NVM 120 may include any suitable type of memory (e.g., magnetic disk, tape, optical disk, etc.) for permanently storing data depending on the implementation of storage device 102.
[0047] The storage controller 116 may be connected to the NVM 120 and the storage memory 118 via one or more storage interfaces 122a, 122b (collectively referred to as 122). The storage controller 116 may receive input / output (I / O) requests (e.g., load or store requests) from the host 100, and send commands to and from the NVM 120 and / or the storage memory 118 to implement the I / O requests. In this regard, the storage controller 116 may include at least one processing component embedded thereon for interfacing with the host 100, the storage memory 118, and the NVM 120. The processing component may include, for example, a digital circuit (e.g., a microcontroller, a microprocessor, a digital signal processor, or a logic device (e.g., a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.)) capable of executing data access instructions (e.g., via firmware and / or software) to provide access to data stored in the storage memory 118 or the NVM 120 according to the data access instructions.
[0048] In some embodiments, storage controller 116 receives and processes speculative and regular memory read commands from host 100. Storage controller 116 may process speculative memory reads as prefetch hints. In this regard, storage controller 116 may retrieve data from NVM 120 into storage memory 118 in response to the speculative memory reads.
[0049] In some embodiments, storage controller 116 receives a regular memory read after a speculative memory read. The regular memory read may be for the same address as the speculative memory read. The data requested by the regular memory read may be retrieved from storage memory 118 instead of NVM 120, which helps reduce memory access latency and improve overall system responsiveness.
[0050] In some embodiments, a speculative memory read is a non-blocking command that allows the processor 106 to issue the command and move on to other processing tasks without waiting for the storage device 102 to return data in response to the command. Speculative memory reads can thus help prevent the host's processor 106 from idly waiting for data to return (as is the case with conventional memory reads).
[0051] Figure 2A block diagram of a storage controller 116 is depicted in accordance with one or more embodiments. The storage controller 116 includes an NVM controller 200, a cache controller 202, a storage prefetch engine ("prefetcher") 204, and a prefetch request queue 206. Although the NVM controller 200, the cache controller 202, and the storage prefetch engine 204 are assumed to be separate components, those skilled in the art will recognize that one or more components may be combined or integrated into a single component, or further subdivided into further subcomponents, without departing from the spirit and scope of the inventive concept.
[0052] In some embodiments, NVM controller 200, cache controller 202, and / or storage prefetch engine 204 may include, for example, digital circuits (e.g., microcontrollers, microprocessors, digital signal processors, or logic devices (e.g., field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc. (collectively referred to as processors)). The digital circuits may include a memory that stores instructions (e.g., software, firmware, and / or hardware code) executed by the processor.
[0053] In some embodiments, the NVM controller 200 is configured to receive a data access request from the host 100. Using CXL as an example, the data access request may be a disk request based on the CXL.io protocol, which allows the NVM 120 to be accessed by the host CPU as a disk storage. In some embodiments, the NVM controller 200 includes a flash translation layer (FTL) 208 that receives the data access request and interfaces with the NVM 120 to read data from the NVM and write data to the NVM. In this regard, the FTL 208 may convert a disk block address included in the data access request into a flash block address. In doing so, the FTL 208 may perform wear leveling to move data around the storage cells of the NVM 120, thereby evenly distributing writes to the NVM 120.
[0054] Referring again to CXL as an example, the data access request may be a memory request based on the CXL.mem or CXL.cache protocol, which allows the NVM 120 to be accessed by the host CPU as a device attached memory. In some embodiments, the memory access request is a speculative memory read. The speculative memory read may identify a memory address (e.g., a logical block address (LBA)). The cache controller 202 may forward the memory address to the storage prefetch engine 204. The storage prefetch engine 204 may be configured to manage a prefetch request queue 206. The memory address may be stored in the prefetch request queue 206 (e.g., in a queue location marked by a tail pointer).
[0055] In some embodiments, the storage prefetch engine 204 is configured to retrieve a memory address from a prefetch request queue 206, wherein the memory address identifies data to be prefetched from the NVM 120 to the storage memory 118. The memory address can be retrieved from the location of the prefetch request queue 206 marked by the head pointer. In some embodiments, the storage prefetch engine 204 communicates with the cache controller 202 to retrieve the data stored in the memory address. In this regard, the cache controller 202 can communicate with the NVM controller 200 for accessing and returning data from the NVM 120. In this regard, the cache controller 202 or the FTL 208 can be configured to convert the requested memory block address into a flash memory block address. The NVM controller 200 can retrieve the data from the flash memory block address and forward the data to the cache controller 202. The cache controller can select a cache address (e.g., a cache line 210) to store the data and store the data in the cache address.
[0056] Figure 3 A conceptual diagram of a CPU prefetch engine 114 employing different prefetch strategies based on information stored in a TLB 126 according to one or more embodiments is depicted. In some embodiments, the TLB 126 includes a mapping of a virtual address 300 to a physical address 302. In some embodiments, the TLB 126 is extended to include a prefetch strategy bit (e.g., set) 304, which can determine the type of prefetch strategy to be employed. For example, if the prefetch strategy bit is set, a speculative prefetch strategy can be used, and if the prefetch strategy is not set, a conventional prefetch strategy can be used.
[0057] In some embodiments, the CPU prefetch engine 114 identifies a memory address (e.g., a virtual address) from which data is to be prefetched. The memory address may be identified by a software-based prefetch instruction that is inserted by a compiler into a program executed by the application 112. The software-based prefetch instruction may be the same regardless of the type of prefetch strategy to be employed or the memory device from which data is to be prefetched. In some embodiments, the address to be prefetched may be identified by the CPU prefetch engine 114 or a separate hardware prefetcher (not shown) via a hardware mechanism.
[0058] In some embodiments, the CPU prefetch engine 114 determines a physical address 302 associated with the identified virtual address 300. The CPU prefetch engine 114 may further retrieve a speculative prefetch policy bit 304 corresponding to the virtual and physical addresses. If the prefetch policy bit 304 is set (e.g., a value of "1"), the CPU prefetch engine 114 may issue a speculative read command (e.g., to the storage device 102) for the identified physical address, followed by a regular memory read command for the same physical address. If the prefetch policy bit 304 is not set (e.g., a value of "0"), the CPU prefetch engine 114 may issue a regular read command (e.g., to the main memory 108) for the identified physical address.
[0059] although Figure 3 108, but a separate table may be used to map memory addresses to host accessible memory devices (e.g., storage device 102 or main memory 108) to allow the CPU prefetch engine 114 to issue commands to the appropriate memory device (e.g., storage device 102 or main memory 108). In this regard, the prefetch policy bits 304 indicate the type of memory read command to be issued (e.g., speculative or conventional), and do not need to distinguish between different types of memory devices. Therefore, in the case where the latency of the storage device 102 is the same as the latency of the main memory 108, the prefetch policy of the memory address in the storage device 102 can be set to "0" to indicate that a conventional prefetch policy is used.
[0060] In some embodiments, the prefetch policy bits 304 are populated by the operating system (e.g., during page table initialization). The prefetch policy bits 304 may be determined based on performance characteristics of the memory devices 102, 108. For example, based on determining that the access latency of the memory device is above a threshold, a speculative prefetch policy may be selected for a memory address in the memory device 102. Based on determining that the access latency of the main memory is less than a threshold, a conventional prefetch policy may be selected for a memory address in the main memory 108.
[0061] Figure 4 An example data flow for prefetching data from a storage device 102 is depicted in accordance with one or more embodiments. A program executed by an application 112 may include a prefetch instruction 400 for a block of data X. The prefetch instruction 400 may cause the host 100 (e.g., via the CPU prefetch engine 114) to send a speculative memory read 402 to the storage device 102. In some embodiments, the speculative memory read is a non-blocking memory read that allows a processor 106 in the host 100 to issue a read command without waiting for data to be returned in response to the command. Thus, the processor 106 is free to execute other instructions after sending the speculative read command.
[0062] In some embodiments, the storage device 102 interprets the speculative read command as a prefetch hint. In this regard, the storage device 102 may issue a memory read 404 to the storage memory 118. In the event that the memory read results in a cache miss 406, the storage device 102 retrieves the data from the NVM 120 and stores the data in the storage memory 118.
[0063] When a load request 408 for data block X is issued by a program executed by an application (or when the CPU prefetch engine 114 issues a regular memory read following a speculative memory read for prefetch purposes), the host 100 may issue a regular memory read 410 to the storage device 102. The storage device may issue a memory read 412 to the storage memory 118. Because the requested data has already been prefetched to the storage memory 118, a cache hit results, allowing the data to be returned with lower latency than when the data is retrieved from the NVM 120.
[0064] Figure 5 A flow chart of a process for prefetching data from a storage device 102 into a CPU cache 124 according to one or more embodiments is depicted. The process begins, and in act 500, the CPU prefetch engine 114 identifies a memory address (e.g., a first address) storing data to be prefetched (e.g., first data). The address may be included in a prefetch instruction included in a program executed by an application 112 running on a processor 106 of a host. In some embodiments, the prefetch instruction may be the same regardless of the prefetch strategy to be employed or the memory device from which the data is to be prefetched. In some embodiments, the memory address is a virtual memory address. In embodiments where the processor includes a hardware prefetch engine, the address to be prefetched is provided by the hardware prefetch engine.
[0065] In action 502, the CPU prefetch engine 114 identifies a prefetch policy (e.g., a first setting) associated with the address. In this regard, the CPU prefetch engine 114 accesses the TLB 126 to determine the value of the prefetch policy bit 304 of the address to be prefetched. If the prefetch policy bit 304 is set to "1", the CPU prefetch engine 114 can determine that the prefetch policy to be used is a speculative prefetch policy.
[0066] In action 504, the CPU prefetch engine 114 issues a first command to the appropriate memory device based on the identified prefetch policy bits 304. For example, if the identified prefetch policy is a speculative prefetch policy, the CPU prefetch engine 114 may issue a speculative memory read command (e.g., via the CXL.mem protocol). In some embodiments, the storage device 102 receives the speculative read command and retrieves the first data associated with the address from the NVM 120 (e.g., the first storage medium) to the storage memory 118 (e.g., the second storage medium). In some embodiments, the storage device 102 may first check the storage memory 118 to determine whether the data to be prefetched already exists. Because the speculative memory read command is a non-blocking command, the CPU is free to execute other instructions without waiting to receive a response from the memory device.
[0067] In act 506, if the identified prefetch policy is a speculative prefetch policy, the CPU prefetch engine 114 may issue a regular read command for the memory address as a second command (e.g., via the CXL.mem protocol). The storage device 102 may process the read command and return the requested data to the host 100. In some embodiments, because the speculative read command in act 504 allows data to be retrieved from the NVM 120 into the storage memory 118, the regular read command allows data to be retrieved from the storage memory 118 in a faster manner than retrieving data from the NVM.
[0068] In act 508, the retrieved data is stored in the CPU cache 124, allowing the data to be quickly returned to the requesting program when the program requests the data (eg, to perform a computational function).
[0069] Figure 6 A flow chart of a process for processing a memory read command from a host 100 according to one or more embodiments is depicted. In act 600, a storage device 102 (e.g., cache controller 202) receives a speculative memory read command. In some embodiments, cache controller 202 is configured to interpret the speculative memory read command as a prefetch hint.
[0070] In act 602, the memory address is retrieved from the command and provided to, for example, the memory prefetch engine 204 for storage in the prefetch request queue 206. The memory prefetch engine 204 may be configured to identify a tail pointer of the queue and insert the address at the location identified by the tail pointer.
[0071] The storage prefetch engine 204 may also be configured to process an address from the prefetch request queue 206 to prefetch data stored at the address. In action 604, the storage prefetch engine 204 retrieves the address at the head of the queue. The address may be provided to the cache controller 202 for prefetching. In some embodiments, before retrieving data from the NVM 120, the cache controller 202 determines whether the data stored at the address is already located in the storage memory 118. If the data is already stored in the storage memory 118, the cache controller 202 may ignore or discard the prefetch address.
[0072] If the data is not already stored in storage memory 118, then in act 606, cache controller 202 may communicate with NVM controller 200 to retrieve the data from NVM 120. In some embodiments, FTL 208 in NVM controller 200 may identify a flash block address associated with the address retrieved from prefetch request queue 206. The data may then be retrieved from the flash block address.
[0073] In act 608, cache controller 202 stores the data retrieved from NVM 120 into storage memory 118. In this regard, cache controller 202 identifies a cache line 210 in which to store the data and stores the data in the identified cache line.
[0074] In act 610, cache controller 202 receives a regular memory read request for data from host 100. Cache controller 202 checks storage memory 118 to determine if the data is in storage memory 118 (e.g., a cache hit). In act 612, the data is retrieved from storage memory 118 and provided to host 100 for storage in, for example, CPU cache 124.
[0075] It should be understood that the systems and methods for prefetching data described herein provide various advantages, including: 1) they allow prefetch hints to be sent to storage devices using prefetch instructions; 2) they help reduce memory access latency and improve overall system performance and responsiveness due to reduced memory access latency; 3) no special operating system, runtime support, or data structures are required to implement data prefetching; 4) the same prefetch instructions that can be used by the compiler can be used to prefetch data via speculative memory reads; and 5) existing compiler optimizations and heuristics that are already in place for prefetching data can be used to prefetch data using speculative memory reads.
[0076] One or more embodiments of the present disclosure may be implemented in one or more processors. The term processor may refer to one or more processors and / or one or more processing cores. One or more processors may be hosted in a single device or distributed across multiple devices (e.g., on a cloud system). The processor may include, for example, an application specific integrated circuit (ASIC), a general or dedicated central processing unit (CPU), a digital signal processor (DSP), a graphics processing unit (GPU), and a programmable logic device such as a field programmable gate array (FPGA). As used herein, in a processor, each function is either performed by hardware configured to (i.e., hardwired) perform the function, or by more general hardware (such as a CPU) configured to execute instructions stored in a non-temporary storage medium (e.g., a memory). The processor may be manufactured on a single printed circuit board (PCB) or distributed on several interconnected PCBs. The processor may include other processing circuits; for example, the processing circuit may include two processing circuits interconnected on a PCB, an FPGA and a CPU.
[0077] It should be understood that although the terms "first", "second", "third", etc. may be used herein to describe various elements, components, regions, layers and / or parts, these elements, components, regions, layers and / or parts should not be limited by these terms. These terms are only used to distinguish one element, component, region, layer or part from another element, component, region, layer or part. Therefore, without departing from the spirit and scope of the present invention, the first element, first component, first region, first layer or first part discussed herein may be referred to as the second element, second component, second region, second layer or second part.
[0078] The terms used herein are only used for the purpose of describing specific embodiments and are not intended to limit the concept of the present invention. In addition, unless explicitly stated, the embodiments described herein are not mutually exclusive. The various aspects of the embodiments described herein can be combined in some implementations.
[0079] As used herein, the terms "substantially," "approximately," and similar terms are used as terms of approximation rather than terms of degree, and are intended to account for the inherent variations in measurements or calculations that one of ordinary skill in the art would recognize.
[0080] As used herein, the singular forms "one" and "an" are also intended to include plural forms, unless the context clearly indicates otherwise. It will also be understood that the terms "including" and / or "comprising" when used in this specification specify the presence of features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their combinations. As used herein, the term "and / or" includes any and all combinations of one or more related listed items. When located in front of the element list, expressions such as "at least one" modify the entire element list without modifying the individual elements of the list. In addition, when describing the embodiments of the present invention, the use of "may" refers to "one or more embodiments of the present disclosure". In addition, the term "exemplary" is intended to refer to an example or description. As used herein, the terms "use", "being used" and "used" may be considered to be synonymous with the terms "utilize", "being utilized" and "utilized", respectively.
[0081] Although exemplary embodiments of systems and methods for prefetching data have been specifically described and illustrated herein, many modifications and variations will be apparent to those skilled in the art. Therefore, it should be understood that systems and methods for prefetching data constructed according to the principles of the present disclosure may be implemented in a manner different from that specifically described herein. The present disclosure is also defined in the following claims and their equivalents.
[0082] Systems and methods for processing storage transactions may include one or more combinations of the features set forth in the following statements.
[0083] Statement 1. A computing system comprising: a storage device including a first storage medium and a second storage medium; a processor configured to communicate with the storage device; a memory coupled to the processor, the memory storing instructions that, when executed by the processor, cause the processor to: identify a first address; identify a first setting associated with the first address; and issue a first command to the first storage medium based on the first setting, wherein the first command is for performing a first type of memory read, wherein the storage device is configured to retrieve first data associated with the first address in the first storage medium to the second storage medium based on the first command.
[0084] Statement 2. The computing system of Statement 1, wherein the first address is identified in a prefetch instruction included in a program executed by the processor.
[0085] Statement 3. The computing system of Statement 1, wherein the first storage medium comprises non-volatile memory and the second storage medium comprises volatile memory.
[0086] Statement 4. The computing system of Statement 1, wherein the first type of memory read comprises: issuing a first command to read data in a first address, wherein the processor is configured to issue a second command for a second function during an interval pending the first command.
[0087] Statement 5. A computing system as described in Statement 1, wherein the instructions further cause the processor to: identify a second address; identify a second setting associated with the second address; and issue a second command to the memory based on the second setting, wherein the second command is used to perform a second type of memory read different from the first type of memory read, and wherein the memory is configured to retrieve second data associated with the second address based on the second command.
[0088] Statement 6. The computing system of Statement 5, wherein the second type of memory read comprises a blocking memory read of the second address.
[0089] Statement 7. The computing system of Statement 1, wherein the storage device includes a queue for storing the first address.
[0090] Statement 8. The computing system of Statement 7, wherein the storage device includes a controller configured to: retrieve the first address from the queue; retrieve the first data from the first storage medium based on the first address retrieved from the queue; and store the first data to the second storage medium.
[0091] Statement 9. The computing system of Statement 1, wherein the first setting identifies a first prefetch strategy, wherein the first prefetch strategy is based on an access latency associated with the first storage device.
[0092] Statement 10. The computing system of Statement 1, wherein the first setting is stored in a table containing a mapping of virtual addresses to physical addresses.
[0093] Statement 11. A method comprises: identifying, by a processor in communication with a storage device, a first address; identifying, by the processor, a first setting associated with the first address; and issuing, by the processor, a first command to a first storage medium of the storage device based on the first setting, wherein the first command is used to perform a first type of memory read, wherein the storage device is configured to retrieve first data associated with the first address in the first storage medium to a second storage medium of the storage device based on the first command.
[0094] Statement 12. The method of Statement 11, wherein the first address is identified in a prefetch instruction included in a program executed by the processor.
[0095] Statement 13. The method of Statement 11 wherein the first storage medium comprises non-volatile memory and the second storage medium comprises volatile memory.
[0096] Statement 14. The method of Statement 11, wherein the first type of memory read comprises: issuing a first command to read data in a first address, wherein the processor is configured to issue a second command for a second function during an interval in which the first command is pending.
[0097] Statement 15. The method according to Statement 11 further includes: identifying a second address; identifying a second setting associated with the second address; and issuing a second command to a memory coupled to the processor based on the second setting, wherein the second command is used to perform a second type of memory read different from the first type of memory read, wherein the memory retrieves second data associated with the second address based on the second command.
[0098] Statement 16. The method of Statement 15, wherein the second type of memory read comprises a blocking memory read of the second address.
[0099] Statement 17. The method of Statement 11 wherein the storage device comprises a queue for storing the first address.
[0100] Statement 18. The method of statement 17, wherein the storage device includes a controller, wherein the controller: retrieves the first address from the queue; retrieves the first data from the first storage medium based on the first address retrieved from the queue; and stores the first data to the second storage medium.
[0101] Statement 19. The method of Statement 11, wherein the first setting identifies a first prefetch strategy, wherein the first prefetch strategy is based on an access latency associated with the first storage device.
[0102] Statement 20. The method of Statement 11, wherein the first setting is stored in a table containing a mapping of virtual addresses to physical addresses.
Claims
1. A computing system for pre-fetching data, include: Storage devices, including: a first storage medium, and a second storage medium; a processor configured to communicate with a storage device; A memory, coupled to the processor, the memory storing instructions that, when executed by the processor, cause the processor to: identifying a first address; identifying a first setting associated with the first address; and A first command is issued to the first storage medium based on the first setting, wherein the first command is used to perform a first type of memory read, wherein the storage device is configured to retrieve first data associated with a first address in the first storage medium to a second storage medium based on the first command.
2. The computing system according to claim 1, in, The first address is identified in a prefetch instruction included in a program executed by the processor.
3. The computing system according to claim 1, in, The first storage medium includes a non-volatile memory, and the second storage medium includes a volatile memory.
4. The computing system according to claim 1, in, The first type of memory reads includes: A first command is issued to read data in a first address, wherein the processor is configured to issue a second command for a second function during an interval in which the first command is pending.
5. The computing system according to claim 1, in, The instructions further cause the processor to: identifying a second address; identifying a second setting associated with the second address; and A second command is issued to the memory based on the second setting, wherein the second command is used to perform a second type of memory read different from the first type of memory read, wherein the memory is configured to retrieve second data associated with the second address based on the second command.
6. The computing system according to claim 5, in, The second type of memory read comprises a blocking memory read of the second address.
7. The computing system according to claim 1, in, The storage device includes a queue for storing a first address.
8. The computing system according to claim 7, in, The storage device includes a controller configured to: Retrieve the first address from the queue; retrieving first data from a first storage medium based on a first address retrieved from the queue; as well as The first data is stored in a second storage medium.
9. The computing system according to claim 1, in, The first setting identifies a first pre-fetch strategy, wherein the first pre-fetch strategy is based on an access latency associated with the first storage device.
10. The computing system according to claim 1, in, The first setting is stored in a table containing a mapping of virtual addresses to physical addresses.
11. A method for pre-fetching data include: identifying, by a processor in communication with the storage device, a first address; identifying, by the processor, a first setting associated with the first address; as well as A first command is issued by a processor to a first storage medium of a storage device based on a first setting, wherein the first command is used to perform a first type of memory read, wherein the storage device is configured to retrieve first data associated with a first address in the first storage medium to a second storage medium of the storage device based on the first command.
12. The method according to claim 11, in, The first address is identified in a prefetch instruction included in a program executed by the processor.
13. The method according to claim 11, in, The first storage medium includes a non-volatile memory, and the second storage medium includes a volatile memory.
14. The method according to claim 11, in, The first type of memory reads includes: A first command is issued to read data in a first address, wherein the processor is configured to issue a second command for a second function during an interval in which the first command is pending.
15. The method according to claim 11, further comprising: include: identifying a second address; identifying a second setting associated with a second address; as well as A second command is issued to a memory coupled to the processor based on the second setting, wherein the second command is used to perform a second type of memory read different from the first type of memory read, wherein the memory retrieves second data associated with the second address based on the second command.
16. The method according to claim 15, in, The second type of memory read comprises a blocking memory read of the second address.
17. The method according to claim 11, in, The storage device includes a queue for storing a first address.
18. The method according to claim 17, in, The storage device comprises a controller, wherein the controller: Retrieve the first address from the queue; retrieving first data from the first storage medium based on the first address retrieved from the queue; and The first data is stored in a second storage medium.
19. The method according to claim 11, in, The first setting identifies a first pre-fetch strategy, wherein the first pre-fetch strategy is based on an access latency associated with the first storage device.
20. The method according to claim 11, in, The first setting is stored in a table containing a mapping of virtual addresses to physical addresses.
Citation Information
Patent Citations
Managing processing of memory commands in a memory subsystem with a high latency backing store
CN114003518A
Relocating data to low latency memory
US20200133585A1
