Storage device, system and method for customizable metadata

Abstract

A storage device includes: a protected memory including one or more log pages, a non-volatile memory, and a storage controller. The storage controller includes: a command fetcher, the command fetcher is used to receive a data request command associated with data including first metadata and second metadata, and execute the data request command in the non-volatile memory; a recorder, the recorder is used to identify the second metadata and record the second metadata in the one or more log pages; and a log page fetcher / eraser, the log page fetcher / eraser is used to obtain the second metadata from the one or more log pages in response to a separate command.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefits of U.S. Provisional Application No. 63 / 186,586, filed on May 10, 2021, entitled “CUSTOM SSD,” and U.S. Patent Application No. 17 / 366,584, filed on July 2, 2021, the entire contents of which are incorporated herein by reference. Technical Field

[0003] Aspects of one or more embodiments of the present disclosure relate to a storage device, a storage system, and a method, and more particularly, to a storage device for customizable metadata, a storage system including the storage device, and a method for managing metadata. Background Art

[0004] Typically, metadata provides important information about one or more aspects of data. In the context of storage systems and database systems, metadata may refer to information about data stored in one or more storage devices, and may include, for example, information about data elements, index structures, schemas, mappings, and / or the like. Therefore, metadata management may be critical to the overall performance of storage systems and database systems, and thus, a large portion of the overhead of storage systems and database systems may be devoted to the handling and processing of metadata.

[0005] The above information disclosed in this Background section is for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute prior art. Summary of the invention

[0006] One or more embodiments of the present disclosure relate to a storage device, a storage system, and a method for customizable metadata.

[0007] According to one or more embodiments of the present disclosure, a storage device includes: a protected memory, the protected memory including one or more log pages; a non-volatile memory; and a storage controller, the storage controller including: a command fetcher, the command fetcher is configured to receive a data request command associated with data including first metadata and second metadata, and execute the data request command in the non-volatile memory; a recorder, the recorder is configured to identify the second metadata and record the second metadata in the one or more log pages; and a log page fetcher / eraser, the log page fetcher / eraser is configured to obtain the second metadata from the one or more log pages in response to a separate command.

[0008] In an embodiment, the command fetcher may be further configured to store the first metadata together with the data in the non-volatile memory, and the logger may be further configured to store the second metadata separately in the one or more log pages.

[0009] In an embodiment, the protected memory may be battery backed dynamic random access memory (DRAM).

[0010] In an embodiment, the protected memory may be a DRAM connected to a protection capacitor.

[0011] In an embodiment, the storage device may further include: a log flusher, the log flusher being configured to: transfer the one or more log pages from the protected memory to the non-volatile memory; and transfer log pages from the non-volatile memory to the protected memory.

[0012] In an embodiment, the log flusher may be further configured to: in response to the protected memory exceeding a memory threshold or losing power, transfer the one or more log pages from the protected memory to the non-volatile memory.

[0013] In an embodiment, the logger may also be configured to expose an application programming interface (API) for managing the one or more log pages.

[0014] In an embodiment, the API may include at least one of: a list recorders command for providing a list of available recorders to the requester, a write recorder command for adding a new recorder in the protected memory, and a select recorder command for selecting a recorder.

[0015] In an embodiment, the separate command may include at least one of: a fetch command for returning a log page among the one or more log pages, an erase command for erasing a log page among the one or more log pages, and an iterative command for returning a series of log pages among the one or more log pages.

[0016] According to one or more embodiments of the present disclosure, a storage system includes: one or more processing circuits; and a storage device, which is communicatively connected to the one or more processing circuits, and the storage device includes: a protected memory, the protected memory including one or more log pages; a non-volatile memory; and a storage controller, the storage controller being configured to: receive a data request command to be executed in the non-volatile memory; identify metadata included with the data request command; record the metadata in the one or more log pages; receive a separate command from a requester among the one or more processing circuits; and obtain the metadata from the one or more log pages in response to the separate command.

[0017] In an embodiment, the storage controller may be further configured to: transfer the one or more log pages from the protected memory to the non-volatile memory; and transfer the log pages from the non-volatile memory to the protected memory.

[0018] In an embodiment, the memory controller may be further configured to: in response to the protected memory exceeding a memory threshold or losing power, transfer the one or more log pages from the protected memory to the non-volatile memory.

[0019] In an embodiment, the storage controller may include an application programming interface (API) for enabling the one or more processing circuits to manage the one or more log pages.

[0020] In an embodiment, the API may include at least one of: a list recorders command for providing a list of available recorders to the requester, a write recorder command for adding a new recorder in the protected memory, and a select recorder command for selecting a recorder.

[0021] In an embodiment, the separate command may include at least one of: a fetch command for returning a log page among the one or more log pages to the requester, an erase command for erasing a log page among the one or more log pages, and an iterative command for returning a series of log pages among the one or more log pages.

[0022] According to one or more embodiments of the present disclosure, a method for managing metadata includes: receiving a data request command to be executed in a non-volatile memory; determining metadata included with the data request command; recording the metadata in one or more log pages in a protected memory; receiving a separate command; and obtaining the metadata from the one or more log pages in response to the separate command.

[0023] In an embodiment, the method may further include: transferring the one or more log pages from the protected memory to the non-volatile memory; and transferring log pages from the non-volatile memory to the protected memory.

[0024] In an embodiment, transferring the one or more log pages from the protected memory to the non-volatile memory may include: determining whether the protected memory exceeds a memory threshold; and in response to the memory threshold being exceeded, transferring the one or more log pages from the protected memory to the non-volatile memory.

[0025] In an embodiment, transferring the one or more log pages from the protected memory to the non-volatile memory may include: determining whether the protected memory loses power or reduces power; and in response to the loss of power or the reduction of power, transferring the one or more log pages from the protected memory to the non-volatile memory.

[0026] In an embodiment, the separate command may include at least one of: a fetch command for returning a log page among the one or more log pages, an erase command for erasing a log page among the one or more log pages, and an iterative command for returning a series of log pages among the one or more log pages. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The above and other aspects and features of the present disclosure will be more clearly understood from the following detailed description of illustrative, non-limiting example embodiments with reference to the accompanying drawings.

[0028] Figure 1 A schematic block diagram of a storage system according to one or more embodiments of the present disclosure is illustrated.

[0029] Figure 2 A schematic block diagram illustrating a processing circuit and a storage device included in a storage system according to one or more embodiments of the present disclosure.

[0030] Figure 3 The diagram illustrates a schematic flow of a method for executing a data request command according to one or more embodiments of the present disclosure.

[0031] Figure 4A is a schematic block diagram illustrating an example of data and metadata included with a data request command according to one or more embodiments of the present disclosure.

[0032] Figure 4B is a schematic block diagram illustrating an example of data and metadata included with a data request command according to one or more embodiments of the present disclosure.

[0033] Figure 4C is a schematic block diagram illustrating an example of data and metadata included with a data request command according to one or more embodiments of the present disclosure.

[0034] Figure 4D is a schematic block diagram illustrating an example of data and metadata included with a data request command according to one or more embodiments of the present disclosure.

[0035] Figure 5 A flowchart illustrating a method of executing a write data request command according to one or more embodiments of the present disclosure.

[0036] Figure 6 A flowchart illustrating a method of executing an erase data request command according to one or more embodiments of the present disclosure.

[0037] Figure 7 A flowchart illustrating a method of managing customizable metadata according to one or more embodiments of the present disclosure. DETAILED DESCRIPTION

[0038] Hereinafter, example embodiments will be described in more detail with reference to the accompanying drawings, in which similar reference numerals refer to similar elements throughout. However, the present disclosure may be implemented in various different forms and should not be construed as being limited to the embodiments illustrated herein. On the contrary, these embodiments are provided as examples so that the present disclosure will be comprehensive and complete, and will fully convey the various aspects and features of the present disclosure to those skilled in the art. Therefore, it is not necessary for those of ordinary skill in the art to fully understand the various aspects and features of the present disclosure to describe the processes, elements and techniques. Unless otherwise noted, similar reference numerals throughout the drawings and written descriptions represent similar elements, and therefore, their descriptions may not be repeated.

[0039] Typically, storage system overhead may mainly come from metadata handling. Such metadata overhead may include, for example, inefficient input / output (IO) patterns (e.g., small synchronous IO), update overhead (e.g., indexing), write amplification (e.g., for consistency), and calculations (e.g., sorting, hashing, etc.). For example, creating or processing metadata may include complex processing operations (e.g., string sorting and hashing) or ordered processing, which can cause large delays between IOs and reduce the throughput of the storage system. In addition, for consistency purposes, in-memory metadata may be synchronized with corresponding on-disk structures (e.g., primary index structures), resulting in increased write amplification and multiple small synchronous IOs, which may delay other data request operations until the multiple small synchronous IOs are completed.

[0040] To reduce such high metadata overhead, some performance-oriented storage systems can be configured to delay metadata processing by first safely storing the raw data of the metadata to disk without or with relatively little computation, and then later processing the raw data in the background to generate metadata. As an example, a log-structured merge tree (e.g., RocksDB) can store key-value pairs to temporary files, from which keys can be read later, an index of the keys can be generated, and then the main index structure can be updated. However, even in this case, small synchronous IOs may be used to provide strong consistency, and background processing overhead (e.g., such as compression overhead) may still dominate resources, which can affect the overall throughput of the storage system.

[0041] According to one or more embodiments of the present disclosure, a storage device, storage system, and method for customizable metadata may be provided, wherein customizable metadata (or customizable metadata information) may be recorded in a durable space and may be acquired on demand (e.g., in response to a user command, user configuration, periodically, or as desired) by various suitable collaborative storage services or applications for later processing. For example, a user may send any desired information by piggybacking on any suitable data request command so that the customizable metadata is recorded in a predefined and / or user-defined log page, and the log page may then be retrieved and erased by any suitable collaborative storage service or application as needed or desired.

[0042] In some embodiments, the durable space may include, for example, protected memory (e.g., battery-backed DRAM) of a storage device, and, for example, in response to a threat of being lost (e.g., limited space, power failure, switching power supplies, and / or the like), log pages may be selectively transferred to non-volatile memory (e.g., NAND flash memory) of the storage device. Therefore, if the protected memory has a large enough capacity and the recorded metadata is retrieved and erased quickly enough by a collaborative storage service or application, it may not be necessary to store the log pages in the non-volatile memory, which can reduce synchronous IO caused by the frequent transfer of metadata and can increase the life expectancy of the non-volatile memory. Therefore, consistency can be relatively guaranteed while increasing the throughput of the storage device and the entire storage system.

[0043] In some embodiments, the customizable metadata may be user-defined and the log page may be user-defined, so that the user may be able to record any desired information in the log page, and the user may transmit the customizable metadata (or customizable metadata information) to be recorded in the log page by carrying the customizable metadata with any suitable data request command. For example, the data associated with the data request command may include user data (e.g., data to be read / written / erased, etc.) and metadata information associated with the data request command, such as cyclic redundancy check (CRC) data sent by the user together with the user data. The customizable metadata (or customizable metadata information) to be recorded in the log page may also be included with the data associated with the data request command, and may be recorded in the log page while the data request command is executed. The customizable metadata (or customizable metadata information) may include any suitable user-desired metadata that may be removed, erased, etc., on demand (e.g., in response to a user command or user configuration, periodically, or as desired). Thus, one example difference between metadata associated with data (i.e., user data) (e.g., CRC data) and customizable metadata may be that metadata associated with data may be stored and erased together with the data, and therefore, may not be separately acquired based on a separate command (e.g., a separate user command, a separate signal, or a configuration). In addition, one difference between customizable metadata and internal metadata is that internal metadata may not be sent by a user together with user data, but may be used internally by a component of a storage device, and therefore, may not be acquired based on a separate command. Thus, unlike metadata (e.g., CRC data) that is associated with data of a data access command and is typically stored together with the data, or internal metadata that is used internally by a component of a storage device, customizable metadata may include any desired information to be recorded in one or more log pages, and may be stored separately from data and metadata corresponding to a data request command.

[0044] In some embodiments, the storage device may expose a set of application programming interfaces (APIs) for allowing any cooperating storage service or application to generate a user-defined logging function for recording customizable metadata in a log page, and the API may allow access to the log page on demand to process (e.g., post-process) the customizable metadata as needed or desired. For example, any suitable processing circuitry located within the storage system may generate a logger for a log page, and may fetch and / or erase one or more log pages as needed. In some embodiments, if the storage device is configured with suitable computing power, the storage device may further compute (e.g., sort, hash, translate, convert, etc.) the customizable metadata before recording the custom metadata in the log page and / or before providing the log page to a requester (e.g., request processing circuitry). Thus, the overhead associated with processing the customizable metadata may be offloaded from the host device to another processing circuit located within or communicatively connected to the storage system, and / or to the storage device itself, depending on the configuration of the storage device and the system.

[0045] Therefore, the storage device, storage system and method according to one or more embodiments of the present disclosure can be applied to any suitable collaborative storage service or application, wherein the storage device records customizable, user-defined metadata that can be obtained on demand by the collaborative storage service or application for later processing. Some non-limiting examples of such collaborative storage services or applications include, for example, index metadata recording and collaborative indexing, lost write tracking, software-defined fairness and scheduling, access tracking for analysis, data rebalancing, caching and / or the like.

[0046] Figure 1 A schematic block diagram of a storage system according to one or more embodiments of the present disclosure is illustrated.

[0047] Figure 1 A simplified example of a storage system 100 is shown in which one host device 102 is connected to one storage device 104, but the present disclosure is not limited thereto, and the storage system 100 may include any suitable number of host devices each connected to any suitable number of storage devices. In addition, the storage system 100 may include multiple storage devices that may be connected to the same host device (e.g., host device 102) or different host devices. Therefore, in various embodiments, there may be a central node (e.g., a central processing circuit) that can centrally manage metadata stored in multiple storage devices, or the storage system 100 may be a distributed storage system in which multiple nodes (e.g., multiple processing circuits) can manage metadata stored in the same or different storage devices in the multiple storage devices.

[0048] refer to Figure 1 , the host device 102 may be connected to the storage device 104 through the host interface 106. The host device 102 may issue a data request command (e.g., a write / erase command) to the storage device 104 through the host interface 106, and may receive a response from the storage device 104 through the host interface 106. For example, the response may include the requested data stored in the storage device 104, and / or a notification from the storage device 104 that the corresponding data request command has been successfully executed by the storage device 104.

[0049] The host device 102 may include a host processor 108 and a host memory 110. The host processor 108 may be a processing circuit, such as a general-purpose processor or a central processing unit (CPU) core of the host device 102. The host processor 108 may be connected to other components via an address bus, a control bus, a data bus, and / or the like. The host memory 110 may be considered to be a high-performance main memory (e.g., primary memory) of the host device 102. For example, in some embodiments, the host memory 110 may include (or may be) a volatile memory, such as a dynamic random access memory (DRAM). However, the present disclosure is not limited thereto, and the host memory 110 may include (or may be) any suitable high-performance main memory (e.g., primary memory) replacement for the host device 102 as known to those skilled in the art. For example, in other embodiments, the host memory 110 can be a relatively high performance non-volatile memory, such as NAND flash memory, phase change memory (PCM), resistive RAM, spin-transfer torque RAM (STTRAM), any suitable memory based on PCM technology, memristor technology and / or resistive random access memory (ReRAM), and can include, for example, chalcogenides and / or the like.

[0050] The storage device 104 may be considered as an auxiliary memory that can persistently store data accessible by the host device 102. In this environment, the storage device 104 may include (or may be) a relatively slow memory when compared to the high-performance memory of the host memory 110. For example, in some embodiments, the storage device 104 may be an auxiliary memory of the host device 102, such as, for example, a solid-state drive (SSD). However, the present disclosure is not limited thereto, and in other embodiments, the storage device 104 may include (or may be) any suitable storage device, such as, for example, a magnetic storage device (e.g., a hard disk drive (HDD)), an optical storage device (e.g., a Blu-ray disk drive, a compact disk (CD) drive, a digital versatile disk (DVD) drive, etc.), other kinds of flash memory devices (e.g., a USB flash drive, etc.), and / or the like. In various embodiments, the storage device 104 may conform to a large form factor standard (e.g., a 3.5-inch hard disk drive form factor), a small form factor standard (e.g., a 2.5-inch hard disk drive form factor), an M.2 form factor, an E1.S form factor, and / or the like. In other embodiments, the storage device 104 may conform to any suitable or desired derivative of these form factors. For convenience, the storage device 104 may be described hereinafter in the context of an SSD, but the present disclosure is not limited thereto.

[0051] The storage device 104 may be communicatively connected to the host device 102 via a host interface 106. The host interface 106 may facilitate communication between the host device 102 and the storage device 104 (e.g., using a connector and a protocol). In some embodiments, the host interface 106 may facilitate the exchange of storage requests and responses between the host device 102 and the storage device 104. In some embodiments, the host interface 106 may facilitate the transfer of data to or from a host memory 110 of the host device 102 via the storage device 104. For example, in various embodiments, the host interface 106 (e.g., a connector and its protocol) may include (or may conform to) a small computer system interface (SCSI), non-volatile memory express (NVMe), peripheral component interconnect express (PCIe), remote direct memory access (RDMA) over Ethernet, serial advanced technology attachment (SATA), Fibre Channel, serial attached SCSI (SAS), NVMe over fabric (NVMe-oF), and / or the like. In other embodiments, the host interface 106 (eg, a connector and its protocol) may include (or may conform to) various general purpose interfaces, such as, for example, Ethernet, a universal serial bus (USB), and / or the like.

[0052] In some embodiments, the storage device 104 may include a storage controller 112, a storage memory 114, a non-volatile memory (NVM) 116, and a storage interface 118. The storage memory 114 may be a high-performance memory of the storage device 104, and may include (or may be) a volatile memory, such as, for example, a DRAM, but the present disclosure is not limited thereto, and the storage memory 114 may be any suitable type of high-performance volatile or non-volatile memory. In some embodiments, the storage memory 114 may be, for example, a protected memory, so that in the event of a power failure or power outage, the loss of log pages stored in the protected memory may be prevented or substantially prevented. For example, in some embodiments, the storage memory 114 may include (or may be) a battery-backed DRAM, but the present disclosure is not limited thereto. As another example, the storage memory 114 may be connected to a capacitor of a suitable size (e.g., a protection capacitor) to store (e.g., temporarily store) log pages in the event of a power failure. Hereinafter, the storage memory 114 may also be referred to as a protected memory 114. The NVM 116 may persistently store data received, for example, from the host device 102. NVM 116 may include, for example, NAND flash memory, but the disclosure is not limited in this regard and NVM 116 may include any suitable type of memory (e.g., magnetic disk, tape, optical disk, and / or the like) for persistently storing data depending on the implementation of storage device 104.

[0053] The storage controller 112 may be connected to the NVM 116 via a storage interface 118. In the context of an SSD, the storage interface 118 may be referred to as a flash translation layer (FTL), and may be an interface that the NVM 116 (e.g., NAND flash memory) may use to communicate with a processing component (e.g., the storage controller 112) or other device. Commands such as reset, write enable, control signals, clock signals, and / or the like may be sent via the storage interface 118. In addition, a software interface may be used in conjunction with a hardware element that may be used to test / verify the operation of the storage interface 118. The software may be used to read data via the storage interface 118 and write data to the NVM 116. In addition, the software may include firmware (e.g., for controlling write operations, erase operations, and read operations) that may be downloaded onto the hardware element.

[0054] The storage controller 112 may be connected to the host interface 106 and may manage signaling through the host interface 106. In some embodiments, the storage controller 112 may include an associated software layer for managing the physical connector of the host interface 106. The storage controller 112 may respond to IO requests received from the host device 102 through the host interface 106. The storage controller 112 may also manage the storage interface 118 to control the NVM 116 and provide access to the NVM 116. For example, the storage controller 112 may include at least one processing component embedded thereon for interfacing with the host device 102 and the NVM 116. The processing component may include, for example, a digital circuit (e.g., a microcontroller, a microprocessor, a digital signal processor, or a logic device (e.g., a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), and / or the like)) that is capable of executing (e.g., via firmware and / or software) data access instructions to provide access to data stored in the NVM 116 according to the data access instructions. For example, the data access instructions may correspond to data request commands and may include any suitable data storage and retrieval algorithms (eg, read / write / erase) instructions and / or the like.

[0055] According to one or more embodiments of the present disclosure, the storage controller 112 may receive a data request command (e.g., a write / erase command) and customizable metadata included in the command (e.g., carried by the command) through the host interface 106. The storage controller 112 may store the customizable metadata in a log page of the storage memory (e.g., protected memory) 114 while executing (e.g., in parallel or simultaneously) the data request command. In response to a separate command (e.g., a separate user command, a separate signal, or a configuration), the storage controller 112 may obtain the log page from the storage memory (e.g., protected memory) 114, and may provide the log page to a requester (e.g., a processing circuit).

[0056] Figure 2 A schematic block diagram illustrating a processing circuit 202 and a storage device 104 included in a storage system 100 according to one or more embodiments of the present disclosure.

[0057] refer to Figure 2 , the storage controller 112 of the storage device 104 may include a command (CMD) fetcher 204, a custom recorder 206, a log flusher 208, and a log page fetcher / erase 210. In a brief overview, the CMD fetcher 204 may be configured to fetch and store data via the host interface 106 (e.g., see Figure 1) receives data request commands from the host device 102 and may send a response (e.g., a notification) to the host device 102 in response to executing the data request command. The custom logger 206 may manage predefined and / or user-defined logging functions to log customizable metadata in incoming data request commands to appropriate log pages 212 in the protected memory 114. The log flusher 208 may flush log pages 212 from the protected memory 114 to the NVM 116 and may load log pages from the NVM 116 to the protected memory 114 as needed or desired. The log page fetcher / eraser 210 may read log pages 212 from the protected memory 114 and erase log pages 212 as needed (e.g., in response to a separate command such as a user command, a user configuration, periodically, or as desired). Although Figure 2 An example directional flow of some communications between components of storage device 104 is shown, but the present disclosure is not limited to Figure 2 The arrows in the figure indicate the direction of communication, and the communication between various components can be unidirectional or bidirectional.

[0058] In more detail, the custom logger 206 may include predefined loggers and / or one or more user-defined loggers, which may be created by utilizing a set of APIs exposed by the custom logger 206 to allow any suitable cooperating storage service or application (e.g., the log processing circuit 202) to create a user-defined logging function for logging customizable metadata to a suitable log page in the log pages 212. Some examples of the APIs exposed by the custom logger 206 may include, for example: ListLogger, which provides a list of available loggers; WriteLogger, which adds a new logger, for example, by taking a cross-compiled binary or eBPF (if supported); and SelectLogger, which selects an available logger, but the present disclosure is not limited to these examples.

[0059] As an example, for each data request operation (e.g., write or erase), for example, when the key includes metadata, the predefined logger may log the pair of operation type (e.g., write or erase) and key (or other relevant or desired metadata) in the appropriate log page in log pages 212, but the present disclosure is not limited thereto. The user-defined logging function may correspond to any suitable user-provided function that can be used as a logger. The user-provided function may be provided using any suitable programming language (e.g., eBPF, C / C++, and / or similar languages), which may depend on the architecture of the storage device 104. For convenience, a non-limiting example of a user-defined logging function that can be used to create a user-defined logger is shown in Example 1 below.

[0060] Example 1:

[0061] int logger(void*dest,void*key,int keylength,void*value,intvaluelength):

[0062] Add Timestamp at dest addr and increment dest by the number of byteswritten(Add Timestamp at dest addr and increment dest by the number of bytes written);

[0063] Add keylength at dest addr and increment dest by the number of byteswritten(Add keylength at dest addr and increment dest by the number of bytes written);

[0064] Copy keylength bytes from key to dest addr and increment dest by thenumber of bytes written(Copy keylength bytes from key to dest addr and increment dest by the number of bytes written);

[0065] If valuelength<preconfigured valueloglength(If valuelength < preconfigured valueloglength):

[0066] Add valuelength at dest addr and increment dest by the number ofbytes written(Add valuelength at dest addr and increment dest by the number of bytes written);

[0067] Copy valuelength bytes from value to dest addr and increment dest bythe number of bytes written(Copy valuelength bytes from value to dest addr and increment dest by the number of bytes written);

[0068] Else(Else):

[0069] Add valueloglength at dest addr and increment dest by the number of bytes written (add valueloglength at dest address and increment dest by the number of bytes written);

[0070] Copy valueloglength bytes from value to dest addr and increment destby the number of bytes written (copy valueloglength bytes from value to dest address and increment dest by the number of bytes written);

[0071] Return total bytes written.

[0072] In Example 1, the input may be a key-value pair, the destination (dest) may be a byte array (output), and the return value may be the written bytes.

[0073] The log flusher 208 may implement on-demand paging for the log page 212 in the protected memory 114. For example, the log flusher 208 may evict the log page 212 from the protected memory 114 to the NVM 116 in response to the log page 212 being threatened to be lost (e.g., when a threshold limit of the protected memory 114 is reached, when power is lost, and / or when the power is switched), thereby reducing write amplification caused by frequent transfer of metadata from the protected memory 114 to the NVM 116, and reducing small synchronous IOs. In addition, by reducing write amplification and small synchronous IOs of the NVM 116, the life expectancy of the NVM 116 may be improved. Additionally, log flusher 208 may load log pages 212 from NVM 116 into protected memory 114 as needed or desired (eg, when requested by log page fetcher / erase 210 and / or custom logger 206 ).

[0074] The log page fetcher / eraser 210 may allow any suitable cooperating storage service or application (e.g., log processing circuitry 202) to read and erase the log page 212 in order to process (e.g., post-process) the log page 212. For example, in response to various commands (e.g., user commands, user log page commands, signals from other devices, and / or the like), the log page fetcher / eraser 210 may fetch the log page 212 and return it to the requester (e.g., log processing circuitry 202), erase the log page 212 based on requester requirements, and iterate over the log pages to return log pages within a specific range to the requester, but the present disclosure is not limited to these examples.

[0075] Thus, any suitable processing circuitry 202 (e.g., such as a host processor 108, a local / remote storage server processor, a processing circuit connected to a fabric or host interface 106 (e.g., a programmed switch), and / or the like located within or communicatively connected to the storage system 100) may include (e.g., may be) a log processing circuit 202, the log processing circuit 202 including a logger information preprocessor 214 and / or a log post-processor 216, the logger information pre-processor 214 may pre-process metadata to be logged, and / or communicate (e.g., via an API) with a custom logger 206 to generate one or more user-defined loggers, the log post-processor 216 may communicate with a log page fetcher / eraser 210 to retrieve and process one or more log pages 212 in the protected memory 114 and post-process the logged metadata.

[0076] For example, the host device 102 may carry the customizable metadata in a data request command, and the downstream recorder information preprocessor 214 (e.g., such as a processing circuit connected to the fabric or host interface 106) may receive (or intercept) the data request command, extract the customizable metadata from the data request command, modify (e.g., reformat, convert, add, subtract, etc.) the customizable metadata for storage in the log page 212, and send the data request command to the storage device 104 for storage in the log page 212. As another example, the host device 102 may issue a data request command without the customizable metadata, and the downstream recorder information preprocessor 214 may receive the data request command, carry the customizable metadata in the data request command, and send the data request command to the storage device 104 for storage in the log page 212. As another example, the host device may include a recorder information preprocessor for modifying (e.g., reformat, convert, add, subtract, etc.) the customizable metadata before carrying it in the data request command. Therefore, in various embodiments, the recorder information preprocessor can carry customizable metadata in a data request command, can intercept and modify the customizable metadata carried in a data request command, can intercept a data request command issued by a host device that does not have customizable metadata, and can carry customizable metadata in a data request command, etc.

[0077] Furthermore, in some embodiments, the storage controller 112 may be optionally implemented with sufficient computing power such that the storage controller 112 may pre-process (e.g., sort, hash, translate, convert, etc.) the customizable metadata information before recording it in the log page 212, may add or change the customizable metadata information before recording it in the log page 212, and / or may post-process (e.g., calculate, modify, etc.) the customizable metadata information in the log page 212 before returning the log page 212 to the requester. Thus, depending on the implementation (e.g., architecture) of the storage system 100 and the storage device 104 (e.g., eBPF / FPGA / GPU / E-SSD / fabric attached computing (e.g., programmable switch) and / or the like), the overhead associated with processing the log page 212 may be offloaded to any suitable processing circuit 202 in communication with the storage device 104 and / or to the storage device 104 itself.

[0078] Although Figure 2The logger information pre-processor 214 and the log post-processor 216 are shown to be part of the same log processing circuit 202, but the present disclosure is not limited to this. For example, in various embodiments, each of the processors 214 and 216 can be a general-purpose processor or a special-purpose processor, an application-specific integrated circuit (ASIC), one or more field programmable gate arrays (FPGAs), a group of processing components, or other suitable processing components that are communicatively connected to the storage device 104. Each of the processors 214 and 216 can be integrated within a single device (e.g., a single log processing circuit 202) or distributed across multiple separate systems or devices. In addition, each of the processors 214 and 216 can be an internal processor with respect to the storage system 100, or can be an external processor implemented as part of a cloud-based computing system.

[0079] Figure 3 The diagram illustrates a schematic flow of a method for executing a data request command according to one or more embodiments of the present disclosure. FIG. 4A to FIG. 4D is a schematic block diagram illustrating various examples of data and metadata associated with a data request command according to one or more embodiments of the present disclosure.

[0080] refer to Figure 2 and Figure 3 In some embodiments, the command fetcher 204 may receive a data request command (e.g., from the host device 102) and may execute the data request command. For example, in some embodiments, when the data request command is a write command, the command fetcher 204 may execute the write command so that data associated with the write command and metadata associated with the data (e.g., CRC data) (e.g., also referred to as first metadata hereinafter) are stored in the NVM 116 through the storage interface 118. As another example, in some embodiments, when the data request command is an erase command (or delete command), the command fetcher 204 may execute the erase command by first reading the data associated with the erase command and any first metadata associated with the data from the NVM 116 through the storage interface 118, and then erasing the data and the first metadata. In some embodiments, the command fetcher 204 may send a notification 302 to the requester of the data request command (e.g., the host device 102), for example, the notification 302 indicating that the data request command was successfully executed.

[0081] In some embodiments, customizable metadata (e.g., also referred to as second metadata hereinafter) may be included in the data request command (e.g., carried on the data request command). For example, the data associated with the data request command may also include customizable metadata (e.g., second metadata). In this case, while executing the data request command, the command fetcher 204 may send the command (or data associated with the command or customizable metadata) to the custom recorder 206 (e.g., in parallel or simultaneously), and the custom recorder 206 may store the customizable metadata (e.g., second metadata) in one or more appropriate log pages 212 in the protected memory 114. For example, in various embodiments, the command fetcher 204 may parse the customizable metadata from the data associated with the data request command to send the customizable metadata to the custom recorder 206 for recording in the log page 212, or may send the data request command as a whole or in part to the custom recorder 206 to parse the customizable metadata therefrom. However, the present disclosure is not limited thereto, and in some embodiments, the data request command may include: a write command having a predetermined data field including the second metadata, a write command having an empty data field for data to be written to the NVM 116, a write command having a flag indicating that the second metadata is included, a special command indicating that the second metadata is included, and / or a similar command that allows only customizable metadata to be written to an appropriate log page among log pages in the protected memory 114 without writing any data to the NVM 116. In this case, the command fetcher 204 may send the command or the second metadata parsed therefrom to the custom recorder 206 for recording, but may not store any data or metadata in the NVM 116.

[0082] refer to FIG. 4A to FIG. 4D The metadata (e.g., the first metadata and the second metadata) may be included in the data request command (e.g., carried on the data request command) in various suitable ways. For example, the metadata may be included in the data associated with the data request command itself (e.g., as Figure 4A and Figure 4C ), or the storage device 104 may include separate spaces for data and metadata, such as, for example, separate data buffers and metadata buffers (e.g., as shown in Figure 4B In either case, the metadata space or metadata buffer for data may include at least one of first metadata (e.g., metadata such as CRC data associated with data of a data request command) Metadata 1 (or MD1) and / or second metadata (e.g., customizable metadata) Metadata 2 (or MD2) for separate storage in one or more log pages 212.

[0083] For example, in some embodiments, the size of the metadata space may be defined (e.g., predefined) according to the implementation of the storage device 104, so that a portion of the metadata space may be allocated for the first metadata Metadata 1, and another portion of the metadata space may be allocated for the second metadata Metadata 2. In this case, when the custom logger 206 receives a data request command (or customizable metadata or information corresponding to customizable metadata), for example, from the command fetcher 204, it may store all metadata in the metadata space allocated for the second metadata Metadata 2 in appropriate log pages in the log page 212. In another example, in some embodiments, the storage device 104 may include sufficient computing power, for example, so that the custom logger 206 may identify and extract (e.g., based on a token, a unique identifier, and / or the like) the second metadata Metadata 2 from the metadata space associated with the data request command, which may allow for more flexibility in the amount of customizable metadata (or metadata information) that may be carried on any given data request command.

[0084] Therefore, in various embodiments, while the first metadata Metadata 1 may be metadata (e.g., CRC data) that is generally associated with the data of the data access command and stored together with the data according to, for example, the logical block address (LBA) of the data, the second metadata Metadata 2 may be associated with the LBA of the data, or may not be associated with the LBA of the data, and alternatively, may be stored separately in an appropriate log page in the protected memory regardless of the LBA of the data. Therefore, a separate IO to the NVM 116 for recording the second metadata Metadata 2 may be prevented or reduced, and the throughput of the storage system 100 may be improved. Although CRC data is provided as an example of the first metadata Metadata 1, the present disclosure is not limited thereto, and the first metadata Metadata 1 may include any suitable metadata associated with the data of the data request command, such as, for example, an error correction code, an error correction parity bit, and / or the like.

[0085] Although FIG. 4A to FIG. 4C Various examples of carrying metadata on data in various block type storage devices are shown, but the present disclosure is not limited thereto, and embodiments of the present disclosure may be applied to any suitable type of storage device, such as, for example, an object storage device, a partitioned name space (ZNS) storage device, a key-value storage device, and / or the like. Figure 4DAs shown, in a key-value storage device (e.g., KVSSD), metadata can be defined as a key, a value, or as a portion of a key-value. In this case, the second metadata can include the entire key, where the key and the value can be written together, or the second metadata can include only the key or a portion of the key, where the key can be obtained from the appropriate log page using a separate command (e.g., a separate user command, signal, or configuration) as needed or desired.

[0086] Figure 5 1 is a flowchart illustrating a method for executing a write data request command according to one or more embodiments of the present disclosure. For example, method 500 may be performed by Figure 1 and Figure 2 The storage controller 112 shown is executed. However, the present disclosure is not limited thereto, and the operations shown in the method 500 may be performed by any suitable one or any suitable combination of components and elements of one or more of the above-described example embodiments. In addition, the present disclosure is not limited to Figure 5 The order or number of operations of method 500 shown may be changed to any desired order or number of operations as recognized by one of ordinary skill in the art. For example, in some embodiments, the order may be changed, or method 500 may include fewer operations or additional operations.

[0087] refer to Figure 5 , the method 500 may begin, and the storage device 104 may receive a data request command for data to be written to the NVM 116 at block 505. For example, the data request command may be a write command for data, and the command fetcher 204 may receive a data request command for data to be written to the NVM 116 via the host interface 106 (e.g., see Figure 1 and Figure 2 ) receives a data request command from the host device 102.

[0088] The storage device 104 may identify first metadata (e.g., CRC data or other metadata) associated with the data at block 510, and may write the data and the first metadata in the NVM 116 at block 515. For example, the data request command may include data to be written to the NVM 116, and first metadata (or first metadata information) associated with the data of the data request command, and the first metadata may include metadata associated with the LBA of the data of the data request command to be stored with the data, but the present disclosure is not limited thereto.

[0089] The storage device 104 may determine at block 520 whether the data request command includes second metadata for recording in the log page 212. For example, the second metadata may include customizable metadata that may be associated with the data of the data request command (e.g., the LBA of the data), or may not be associated with the data of the data request command (e.g., the LBA of the data) and stored separately from the data. If the data request command does not include the second metadata (e.g., no at block 520), the storage device 104 (e.g., command fetcher 204) may send a notification to the requester (e.g., host device 102) at block 530 indicating that the data request command was successfully executed, and the method 500 may end.

[0090] On the other hand, in response to the storage device 104 (e.g., the command fetcher 204 and / or the custom logger 206) determining at block 520 that the data request command includes second metadata for recording in the log page 212 (e.g., yes at block 520), the storage device 104 (e.g., the custom logger 206) may store (e.g., may record) the second metadata in the appropriate log page 212 of the protected memory 114 at block 525. For example, the storage device 104 (e.g., the command fetcher 204 and / or the custom logger 206) may determine that any metadata in an allocated portion (e.g., a predefined portion) of the metadata space associated with the data request command is the second metadata to be stored in one or more appropriate log pages 212, or the storage device 104 (e.g., the command fetcher 204 and / or the custom logger 206) may have sufficient computing power configured to identify and extract (e.g., via a token, a unique identifier, and / or the like) the second metadata from the metadata space.

[0091] Once the second metadata is successfully stored (e.g., recorded) in the log page 212 at block 525, the storage device 104 (e.g., command fetcher 204) may send a notification to the requester (e.g., host device 102) associated with the data request command at block 530 indicating that the second metadata is successfully recorded, and the method 500 may end. In some embodiments, because blocks 510 and 515 may be performed in parallel (e.g., simultaneously) with blocks 520 and 525 (e.g., see Figure 3 ), so the command fetcher 204 can send a notification when the data request command is successfully executed, and can send a separate notification when the second metadata is successfully recorded, so that one operation may not cause a bottleneck to another operation, but the present disclosure is not limited to this, and in other embodiments, the command fetcher 204 can wait for both operations to be successfully completed, and then can send a single notification indicating that both operations have been successfully completed.

[0092] Although Figure 5 An example is shown in which the second metadata is included in a data request command associated with some data to be written in NVM 116 (e.g., the second metadata is carried together with a data request command associated with some data to be written in NVM 116), but the present disclosure is not limited thereto. For example, in some embodiments, the data request command may be a write command with a special data field or other special command, so that only the second metadata can be recorded in the appropriate log page 212 without writing any data to NVM 116.

[0093] Figure 6 6. A flowchart illustrating a method for executing an erase data request command according to one or more embodiments of the present disclosure. For example, method 600 may be performed by Figure 1 and Figure 2 The storage controller 112 shown is executed. However, the present disclosure is not limited thereto, and the operations shown in the method 600 may be performed by any suitable one or combination of components and elements of one or more of the above-described example embodiments. In addition, the present disclosure is not limited to Figure 6 The order or number of operations of method 600 shown may be changed to any desired order or number of operations as recognized by one of ordinary skill in the art. For example, in some embodiments, the order may be changed, or method 600 may include fewer operations or additional operations.

[0094] refer to Figure 6 , the method 600 may begin, and the storage device 104 may receive a data request command for data to be erased (e.g., deleted) from the NVM 116 at block 605. For example, the data request command may be an erase command for deleting data stored in the NVM 116, and the command fetcher 204 may receive the data request command through the host interface 106 (e.g., see Figure 1 and Figure 2 ) receives a data request command from the host device 102.

[0095] The storage device 104 may retrieve the data and first metadata associated with the data (e.g., CRC data or other metadata) from the NVM 116 at block 610, and may delete the data and the first metadata at block 615. For example, the storage device 104 (e.g., the command fetcher 204) may read the data and the first metadata associated with the data request command from the NVM 116, and may perform an erase operation (e.g., a delete operation) on the data and the first metadata in the NVM 116. The first metadata (or first metadata information) associated with the data of the data request command may be associated with the LBA of the data of the data request command, but the present disclosure is not limited thereto.

[0096] The storage device 104 may determine at block 620 whether the data request command includes second metadata for recording in the log page 212. For example, the second metadata may include customizable metadata that may be associated with the data associated with the data request command (e.g., the LBA of the data), or may not be associated with the data associated with the data request command (e.g., the LBA of the data). If the data request command does not include the second metadata (e.g., no at block 620), the storage device 104 (e.g., command fetcher 204) may send a notification to the requester (e.g., host device 102) at block 630 indicating that the data request command was successfully executed, and the method 600 may end.

[0097] On the other hand, in response to the storage device 104 (e.g., the command fetcher 204 and / or the custom logger 206) determining at block 620 that the data request command includes second metadata for recording in the log page 212 (e.g., yes at block 620), the storage device 104 (e.g., the custom logger 206) may store (e.g., may record) the second metadata in the appropriate log page 212 of the protected memory 114 at block 625. For example, the storage device 104 (e.g., the command fetcher 204 and / or the custom logger 206) may determine that any metadata in an allocated portion (e.g., a predefined portion) of the metadata space associated with the data request command is the second metadata to be stored in one or more appropriate log pages 212, or the storage device 104 (e.g., the command fetcher 204 and / or the custom logger 206) may have sufficient computing power configured to identify and extract (e.g., via a token, a unique identifier, and / or the like) the second metadata from the metadata space.

[0098] Once the second metadata is successfully stored (e.g., recorded) in the log page 212 at block 625, the storage device 104 (e.g., command fetcher 204) may send a notification indicating that the second metadata is successfully recorded to the requester associated with the data request command (e.g., host device 102) at block 630, and the method 600 may end. In some embodiments, because blocks 610 and 615 may be performed in parallel (e.g., simultaneously) with blocks 620 and 625 (e.g., see Figure 3 ), so the command fetcher 204 can send a notification when the data request command is successfully executed, and can send a separate notification when the second metadata is successfully recorded, so that one operation may not cause a bottleneck to another operation, but the present disclosure is not limited to this, and in other embodiments, the command fetcher 204 can wait for both operations to be successfully completed, and then can send a single notification indicating that both operations have been successfully completed.

[0099] Figure 7 1 is a flowchart illustrating a method for managing customizable metadata according to one or more embodiments of the present disclosure. For example, method 700 may be performed by Figure 1 and Figure 2 The storage controller 112 shown is executed. However, the present disclosure is not limited thereto, and the operations shown in the method 700 may be performed by any suitable one or any suitable combination of components and elements of one or more of the above-described example embodiments. In addition, the present disclosure is not limited to Figure 7 The order or number of operations of method 700 shown may be changed to any desired order or number of operations as recognized by one of ordinary skill in the art. For example, in some embodiments, the order may be changed, or method 700 may include fewer operations or additional operations.

[0100] refer to Figure 7 , method 700 may begin, and storage device 104 may receive, for example, from host device 102 (eg, see Figure 1 ) receives a data request command. The data request command may be as described above. Figure 5 The write command discussed above, or as referenced above Figure 6 The erase command discussed above, therefore, the storage device 104 can be based on the above reference Figure 5 and Figure 6 For example, the storage device 104 (eg, the command fetcher 204 or the custom recorder 206) may execute the data request command in block 710 (eg, corresponding to Figure 5 Box 520 or Figure 6 The metadata information (eg, second metadata) included with the data request command may be identified at block 620 of FIG. 1 and may be received at block 715 (eg, corresponding to Figure 5 Box 525 or Figure 6 At box 625 of , the metadata information (eg, second metadata) is recorded (eg, may be stored) in a log page in the protected memory 114.

[0101] In some embodiments, if the data request command is a read command, only the data associated with the read command and the first metadata associated with the data can be read and returned to the requester of the read command. However, the present disclosure is not limited thereto, and in other embodiments, the second metadata to be stored in one or more suitable log pages 212 can be carried together with the read data request command, similar to the method of the erase data request command described above.

[0102] Subsequently, the storage device 104 (e.g., the command fetcher 204 or the log page fetcher / eraser 210) may receive a separate command (e.g., a user command, a signal, etc.) from the requester (e.g., the log processing circuit 202) at block 720 to fetch (e.g., may read) metadata information (e.g., second metadata) from the log page. For example, the separate command (e.g., the user command or the signal) may be a fetch command or an erase command for fetching or erasing the log page stored in the protected memory 114. The storage device 104 (e.g., the log page fetcher / eraser 210) may fetch (e.g., may read) the log page from the protected memory 114 at block 725, may send the metadata information (e.g., second metadata) to the requester (e.g., the log processing circuit 202) at block 730 (or may delete the metadata information), and the method 700 may end.

[0103] According to one or more embodiments of the present disclosure described above, a storage device, system, and method may be provided, wherein customizable metadata may be included in any suitable data request command (e.g., carried on any suitable data request command) to be stored in one or more appropriate log pages in a protected memory. According to one or more embodiments of the present disclosure, the recorded customizable metadata may subsequently be retrieved from the protected memory for further processing, and the processing of the metadata may be offloaded from the host device to any processing circuit located in or communicatively connected to the storage device, or to the storage device itself (e.g., when configured with sufficient computing power). Therefore, the throughput of the storage system may be improved.

[0104] Although various methods according to one or more embodiments of the present disclosure have been described in terms of various processes having a specific processing order, the present disclosure is not limited thereto. For example, when a specific embodiment can be implemented differently, the specific processing order can be different from the described order. For example, two processes described in succession can be performed simultaneously or substantially simultaneously, can be performed in an order opposite to the described order, or can be performed in an order that does not follow the sequential progression presented in the present disclosure.

[0105] It will be understood that although the terms "first", "second", "third", etc. may be used herein to describe various elements, components, regions, layers and / or parts, these elements, components, regions, layers and / or parts should not be limited by these terms. These terms are used to distinguish one element, component, region, layer or part from another element, component, region, layer or part. Therefore, a first element, component, region, layer or part may be referred to as a second element, component, region, layer or part without departing from the spirit and scope of the present disclosure.

[0106] It will be understood that when an element or layer is referred to as being "on," "connected to," or "coupled to" another element or layer, it can be directly on, directly connected to, or coupled to the other element or layer, or one or more intervening elements or layers may be present. Additionally, it will be understood that when an element or layer is referred to as being "between" two elements or layers, it can be the only element or layer between the two elements or layers, or one or more intervening elements or layers may also be present.

[0107] The terms used herein are to describe specific embodiments and are not intended to limit the present disclosure. Unless the context clearly indicates otherwise, the singular forms "one" and "an" as used herein are intended to also include plural forms. It will be further understood that when the terms "include", "comprise", and "have" are used in this specification, it is indicated that there are stated features, integral bodies, steps, operations, elements and / or components, but it is not excluded that there are or add one or more other features, integral bodies, steps, operations, elements, components and / or their groups. As used herein, the term "and / or" includes any and all combinations of one or more related listed items. For example, the statement "A and / or B" means A, B, or A and B. The statement such as "at least one of ... " modifies the elements of the entire list when it is after the list of elements, without modifying the individual elements of the list. For example, the statement "at least one of a, b or c" indicates: only a, only b, only c, both a and b, both a and c, both b and c, all of a, b and c, or its variation.

[0108] As used herein, the terms "substantially," "about," and similar terms are used as terms of approximation, not as terms of degree, and are intended to account for the inherent variations in measurements or calculations that would be recognized by one of ordinary skill in the art. Additionally, the use of "may" when describing embodiments of the present disclosure refers to "one or more embodiments of the present disclosure." As used herein, the term "using" may be considered synonymous with the term "utilizing."

[0109] The electronic device or electrical device and / or any other related device or component according to the embodiment of the present disclosure described herein can be implemented using any suitable hardware, firmware (e.g., application specific integrated circuit), software, or a combination of software, firmware and hardware. For example, the various components of these devices can be formed on an integrated circuit (IC) chip or on a separate IC chip. In addition, the various components of these devices can be implemented on a flexible printed circuit film, a carrier package (TCP), a printed circuit board (PCB), or formed on a substrate. In addition, the various components of these devices can be processes or threads running on one or more processors that execute computer program instructions in one or more computing devices and interact with other system components to perform various functions described herein. The computer program instructions are stored in a memory, and the memory can be implemented in a computing device using a standard memory device such as a random access memory (RAM). The computer program instructions can also be stored in other non-temporary computer-readable media such as, for example, a CD-ROM, a flash drive, etc. In addition, those skilled in the art should recognize that, without departing from the spirit and scope of the exemplary embodiments of the present disclosure, the functions of various computing devices can be combined or integrated into a single computing device, or the functions of a specific computing device can be distributed on one or more other computing devices.

[0110] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those commonly understood by those skilled in the art to which the present disclosure belongs. It will be further understood that unless explicitly defined as such herein, terms (such as those defined in commonly used dictionaries) should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology and / or this specification, and should not be interpreted in an idealized or overly formal sense.

[0111] Although some example embodiments have been described, it will be readily appreciated by those skilled in the art that various modifications are feasible in the example embodiments without departing from the spirit and scope of the present disclosure. It will be understood that, unless otherwise described, the description of features or aspects within each embodiment should generally be considered to be applicable to other similar features or aspects in other embodiments. Therefore, as will be apparent to those of ordinary skill in the art, unless otherwise specifically indicated, the features, characteristics and / or elements described in conjunction with a particular embodiment may be used alone or in combination with the features, characteristics and / or elements described in conjunction with other embodiments. Therefore, it should be understood that the foregoing is an illustration of various example embodiments and should not be construed as being limited to the specific example embodiments disclosed herein, and that various modifications to the disclosed example embodiments and other example embodiments are intended to be included within the spirit and scope of the present disclosure as defined in the appended claims and their equivalents.

Claims

1. A storage device, comprising: protected memory, the protected memory comprising one or more log pages; Non-volatile memory; as well as A storage controller, wherein the storage controller is configured to: receiving a data request command through a host interface and accessing the non-volatile memory according to the data request command, the data request command including data associated with the data request command, first metadata, and second metadata; identifying the second metadata from the data request command; recording the second metadata in the one or more log pages based on a configuration received via a first interface different from the host interface; receiving a single command via the first interface; as well as In response to the separate command, obtaining the second metadata from the one or more log pages, wherein the first metadata is associated with the data and is processed according to the data request command, and The second metadata is user-defined metadata, which is included in the data request command, stored separately from the data and the first metadata, and processed according to the separate command.

2. The storage device according to claim 1, wherein: The storage controller is configured to store the first metadata together with the data in the non-volatile memory and to store the second metadata separately in the one or more log pages.

3. The storage device according to claim 1, wherein: The protected memory is a battery backed dynamic random access memory.

4. The storage device according to claim 1, wherein: The protected memory is a dynamic random access memory connected to a protection capacitor.

5. The storage device according to claim 1, wherein: The storage controller is further configured to: transferring the one or more log pages from the protected memory to the non-volatile memory; as well as A log page is transferred from the non-volatile memory to the protected memory.

6. The storage device according to claim 5, wherein: The memory controller is configured to transfer the one or more log pages from the protected memory to the non-volatile memory in response to the protected memory exceeding a memory threshold or losing power.

7. The storage device according to claim 1, wherein: The storage controller is configured to expose the first interface, which is different from the host interface, as an application programming interface for managing the one or more log pages.

8. The storage device according to claim 7, wherein: The application programming interface includes at least one of a list recorder command for providing a list of available recorders to a requester, a write recorder command for adding a new recorder in the protected memory, and a select recorder command for selecting a recorder.

9. The storage device according to claim 1, wherein: The separate command includes at least one of: a fetch command for returning a log page among the one or more log pages, an erase command for erasing a log page among the one or more log pages, and an iterate command for returning a series of log pages among the one or more log pages.

10. A storage system, comprising: one or more processing circuits; as well as a storage device, the storage device being communicatively connected to the one or more processing circuits, and the storage device comprising: protected memory, the protected memory comprising one or more log pages; non-volatile memory; and A storage controller, wherein the storage controller is configured to: receiving, via a host interface, a data request command to access the non-volatile memory, the data request command comprising data associated with the data request command, first metadata, and second metadata; identifying the second metadata included with the data request command; recording the second metadata in the one or more log pages based on a configuration received via a first interface different from the host interface; receiving, via the first interface, a separate command from a requester among the one or more processing circuits; and In response to the separate command, obtaining the second metadata from the one or more log pages, wherein the first metadata is associated with the data and is processed according to the data request command, and The second metadata is user-defined metadata, which is included in the data request command, stored separately from the data and the first metadata, and processed according to the separate command.

11. The storage system according to claim 10, wherein: The storage controller is further configured to: transferring the one or more log pages from the protected memory to the non-volatile memory; as well as A log page is transferred from the non-volatile memory to the protected memory.

12. The storage system according to claim 11, wherein: The memory controller is configured to transfer the one or more log pages from the protected memory to the non-volatile memory in response to the protected memory exceeding a memory threshold or losing power.

13. The storage system according to claim 10, wherein: The first interface, distinct from the host interface, includes an application programming interface for enabling the one or more processing circuits to manage the one or more log pages.

14. The storage system according to claim 13, wherein: The application programming interface includes at least one of a list recorder command for providing a list of available recorders to the requester, a write recorder command for adding a new recorder in the protected memory, and a select recorder command for selecting a recorder.

15. The storage system according to claim 10, wherein: The separate command includes at least one of: a fetch command for returning a log page among the one or more log pages to the requester, an erase command for erasing a log page among the one or more log pages, and an iterate command for returning a series of log pages among the one or more log pages.

16. A method for managing metadata, comprising: receiving, by the processing circuit through the host interface, a data request command to access a non-volatile memory, the data request command comprising data associated with the data request command, first metadata, and second metadata; determining, by the processing circuit, second metadata included in the data request command; recording, by the processing circuit, the second metadata in one or more log pages in a protected memory based on a configuration received via a first interface different from the host interface; receiving, by the processing circuit, a separate command via the first interface; as well as acquiring, by the processing circuit in response to the separate command, the second metadata from the one or more log pages, wherein the first metadata is associated with the data and is processed according to the data request command, and The second metadata is user-defined metadata, which is included in the data request command, stored separately from the data and the first metadata, and processed according to the separate command.

17. The method according to claim 16, further comprising: transferring, by the processing circuit, the one or more log pages from the protected memory to the non-volatile memory; as well as A log page is transferred, by the processing circuit, from the non-volatile memory to the protected memory.

18. The method according to claim 17, wherein: Transferring the one or more log pages from the protected memory to the non-volatile memory includes: determining, by the processing circuit, that the protected memory exceeds a memory threshold; and The one or more log pages are transferred, by the processing circuit, from the protected memory to the non-volatile memory in response to the memory threshold being exceeded.

19. The method according to claim 17, wherein: Transferring the one or more log pages from the protected memory to the non-volatile memory includes: determining, by the processing circuit, that the protected memory has lost power or has reduced power; and The one or more log pages are transferred, by the processing circuitry, from the protected memory to the non-volatile memory in response to the loss of power or the reduction of power.

20. The method according to claim 16, wherein: The separate command includes at least one of: a fetch command for returning a log page among the one or more log pages, an erase command for erasing a log page among the one or more log pages, and an iterate command for returning a series of log pages among the one or more log pages.

Images